feat: AI prompt management dashboard and enhanced span inspectors#3244
feat: AI prompt management dashboard and enhanced span inspectors#3244
Conversation
…rride fixes, llm pricing sync
…s filter, show version in generation rows
…rsion/override display, breadcrumb fix
…, streamText, generateObject, toolCall, embed)
|
WalkthroughThis pull request introduces comprehensive prompt management functionality. It adds database tables for prompts and prompt versions with Prisma migrations, implements server-side Estimated code review effort🎯 5 (Critical) | ⏱️ ~120+ minutes 🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment Tip CodeRabbit can use your project's `biome` configuration to improve the quality of JS/TS/CSS/JSON code reviews.Add a configuration file to your project to customize how CodeRabbit runs |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 12
Note
Due to the large number of review comments, Critical, Major severity comments were prioritized as inline comments.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
internal-packages/llm-pricing/src/default-model-prices.json (1)
3967-3971:⚠️ Potential issue | 🟠 MajorWiden the live Gemini prefix too.
This is the only
gemini*entry in the catalog that still rejectsgoogleai/...— all other 16 Gemini models use the widenedgoogle(ai)?/pattern. If callers resolve the live model with that prefix, pricing will miss. Update the pattern and re-runpnpm run sync-prices.Suggested fix
- "matchPattern": "(?i)^(google\/)?(gemini-live-2.5-flash-native-audio)$", + "matchPattern": "(?i)^(google(ai)?\/)?(gemini-live-2.5-flash-native-audio)$",🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@internal-packages/llm-pricing/src/default-model-prices.json` around lines 3967 - 3971, Update the matchPattern for the entry with id "029e6695-ff24-47f0-b37b-7285fb2e5785" (modelName "gemini-live-2.5-flash-native-audio") to accept the widened live Gemini prefix used by other Gemini entries: replace the current "(?i)^(google\/)?(gemini-live-2.5-flash-native-audio)$" pattern with one that allows both "google/" and "googleai/" prefixes (matching the other Gemini entries), then re-run "pnpm run sync-prices" to propagate the change.apps/webapp/app/components/metrics/QueryWidget.tsx (1)
452-458:⚠️ Potential issue | 🟠 MajorFullscreen table ignores hidden column configuration.
Line 452-458 renders
TSQLResultsTablewithouthiddenColumns, so hidden fields reappear when maximized.Proposed fix
<TSQLResultsTable rows={data.rows} columns={data.columns} prettyFormatting={config.prettyFormatting} sorting={config.sorting} showHeaderOnEmpty={showTableHeaderOnEmpty} + hiddenColumns={hiddenColumns} />🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/components/metrics/QueryWidget.tsx` around lines 452 - 458, The fullscreen TSQLResultsTable render is missing the hiddenColumns prop so columns configured as hidden reappear when maximized; update the TSQLResultsTable usage in QueryWidget (the block rendering TSQLResultsTable) to pass the hiddenColumns from the current config (e.g., hiddenColumns={config.hiddenColumns}) so the table respects hidden field configuration in both normal and fullscreen modes.
🟡 Minor comments (10)
apps/webapp/app/components/runs/v3/ai/AIEmbedSpanDetails.tsx-74-79 (1)
74-79:⚠️ Potential issue | 🟡 MinorFix minute/second formatting to avoid
60soutputs.At Line 78,
.toFixed(0)can round up and produce invalid displays like1m 60s. Use floor/truncation for seconds.Proposed fix
function formatDuration(ms: number): string { if (ms < 1000) return `${Math.round(ms)}ms`; if (ms < 60_000) return `${(ms / 1000).toFixed(1)}s`; const mins = Math.floor(ms / 60_000); - const secs = ((ms % 60_000) / 1000).toFixed(0); + const secs = Math.floor((ms % 60_000) / 1000); return `${mins}m ${secs}s`; }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/components/runs/v3/ai/AIEmbedSpanDetails.tsx` around lines 74 - 79, The formatDuration function can produce "1m 60s" because secs is computed with toFixed(0) which can round up; update formatDuration to compute seconds with truncation/flooring instead (e.g., secs = Math.floor((ms % 60_000) / 1000)) so seconds never equal 60, and return the string using that secs value; change occurs in the formatDuration function where secs is currently set with .toFixed(0).internal-packages/clickhouse/src/llmMetrics.ts-41-41 (1)
41-41:⚠️ Potential issue | 🟡 MinorConstrain
prompt_versionto an integer.
prompt_versioncurrently accepts any number; allowing decimals here can create invalid prompt-version semantics in stored metrics.Proposed fix
- prompt_version: z.number(), + prompt_version: z.number().int().nonnegative(),🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@internal-packages/clickhouse/src/llmMetrics.ts` at line 41, The prompt_version field in the Zod schema is currently z.number(), which allows non-integer values; update the prompt_version schema to only accept integers (e.g., replace z.number() with z.number().int() or add a refine that checks Number.isInteger) so stored metrics can't have decimal prompt versions; change the prompt_version entry in the llmMetrics schema accordingly.apps/webapp/app/components/code/TSQLResultsTable.tsx-1049-1052 (1)
1049-1052:⚠️ Potential issue | 🟡 MinorHandle “all columns hidden” using
visibleColumnsto avoid blank rendering.After introducing
hiddenColumns, table guards still usecolumns.length. If all columns are hidden, the component can render an unusable empty structure.💡 Suggested fix
- if (!columns.length) return null; + if (!visibleColumns.length) { + return <ChartBlankState icon={IconTable} message="No columns to display" />; + } ... - <td className="w-full px-3 py-6" colSpan={columns.length}> + <td className="w-full px-3 py-6" colSpan={visibleColumns.length}>Also applies to: 1106-1106, 1153-1153
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/components/code/TSQLResultsTable.tsx` around lines 1049 - 1052, The component's rendering guards still check columns.length instead of the computed visibleColumns, so if all columns are hidden the table renders empty; update every guard and conditional that currently uses columns.length to use visibleColumns.length (the useMemo result) and add an explicit early return or fallback UI when visibleColumns.length === 0; search for occurrences around where visibleColumns is used (e.g., in TSQLResultsTable render/guard logic and the other two spots noted) and replace those checks so the component does not render an empty table when all columns are hidden.apps/webapp/app/components/runs/v3/ai/AIModelSummary.tsx-63-66 (1)
63-66:⚠️ Potential issue | 🟡 MinorKeep telemetry
promptfallback when structured prompt fields are missing.Filtering out
promptunconditionally can hide prompt data for older spans that only populatetelemetryMetadata.prompt.💡 Suggested fix
- {aiData.telemetryMetadata && - Object.entries(aiData.telemetryMetadata) - .filter(([key]) => key !== "prompt") - .map(([key, value]) => <MetricRow key={key} label={key} value={value} />)} + {aiData.telemetryMetadata && + Object.entries(aiData.telemetryMetadata) + .filter(([key]) => !(key === "prompt" && aiData.promptSlug)) + .map(([key, value]) => <MetricRow key={key} label={key} value={value} />)}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/components/runs/v3/ai/AIModelSummary.tsx` around lines 63 - 66, Currently the code unconditionally filters out the "prompt" key from aiData.telemetryMetadata, which removes prompt fallback for older spans; modify the logic so "prompt" is only excluded when structured prompt fields exist—e.g., compute hasStructuredPrompt = Object.keys(aiData.telemetryMetadata).some(k => k !== "prompt" && k.startsWith("prompt")) (or check for known structured keys) and then only filter out "prompt" when hasStructuredPrompt is true; keep the mapping to MetricRow unchanged so MetricRow still receives telemetry entries including the fallback "prompt" when no structured prompt fields are present.apps/webapp/app/routes/api.v1.prompts.$slug.override.ts-27-31 (1)
27-31:⚠️ Potential issue | 🟡 MinorKeep the 405 responses aligned with the actual method set.
The loader's non-
OPTIONSpath drops the CORS wrapper andAllowheader entirely, and the action advertisesPOST, PUT, DELETEeven thoughPATCHis accepted above. Unsupported-method clients will get misleading metadata.Also applies to: 113-116
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/routes/api.v1.prompts`.$slug.override.ts around lines 27 - 31, The loader's non-OPTIONS branch returns a bare 405 without CORS or an accurate Allow header; update loader (export async function loader) to wrap the 405 response with apiCors and include an Allow header that matches the actual supported methods (include PATCH alongside POST, PUT, DELETE as accepted elsewhere), and likewise fix the action's 405 paths (the branch around lines ~113-116) so they also use apiCors and the same correct Allow header to avoid misleading metadata.apps/webapp/app/components/runs/v3/PromptSpanDetails.tsx-38-43 (1)
38-43:⚠️ Potential issue | 🟡 MinorReset the active tab when the available sections change.
tabpersists across prop changes. If someone is on "Input" or "Template" and then opens a prompt span without that section, the header renders with no active tab and the body stays blank until they click "Overview".Suggested fix
-import { lazy, Suspense, useState } from "react"; +import { lazy, Suspense, useEffect, useState } from "react"; @@ const [tab, setTab] = useState<PromptTab>("overview"); + + useEffect(() => { + if ((tab === "input" && !hasInput) || (tab === "template" && !hasTemplate)) { + setTab("overview"); + } + }, [tab, hasInput, hasTemplate]);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/components/runs/v3/PromptSpanDetails.tsx` around lines 38 - 43, The active tab stored in state (tab via useState and setTab) can become invalid when availableTabs changes; add a useEffect in the PromptSpanDetails component that watches availableTabs and, if the current tab is not included in availableTabs, calls setTab(availableTabs[0]) (or another sane default) to reset to a valid tab; ensure the effect lists availableTabs (or a stable derived key) in its dependency array so the active tab updates whenever availableTabs changes.packages/cli-v3/src/mcp/tools/prompts.ts-50-61 (1)
50-61:⚠️ Potential issue | 🟡 MinorAdd error handling for non-JSON error responses.
If the API returns a non-JSON error response (e.g., 500 with HTML),
res.json()will throw before the error message can be extracted. Consider handling this case.🛡️ Proposed fix for safer error handling
async function fetchPromptApi( apiClient: { fetchClient: typeof fetch; baseUrl: string }, path: string, options?: RequestInit ) { const res = await apiClient.fetchClient(`${apiClient.baseUrl}/api/v1/prompts${path}`, options); - const data = await res.json(); if (!res.ok) { - throw new Error(data.error ?? `API error ${res.status}`); + let errorMessage = `API error ${res.status}`; + try { + const data = await res.json(); + if (data.error) errorMessage = data.error; + } catch { + // Response wasn't JSON, use status code message + } + throw new Error(errorMessage); } - return data; + return res.json(); }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/cli-v3/src/mcp/tools/prompts.ts` around lines 50 - 61, The fetchPromptApi function currently calls res.json() unconditionally which will throw on non-JSON responses; update fetchPromptApi to first read the response body safely (e.g., await res.text()), then if res.ok try to parse JSON (JSON.parse) and return it, and if !res.ok attempt to parse the text as JSON to extract data.error but fall back to including the raw text and status in the thrown Error; reference the existing fetchPromptApi function and apiClient.fetchClient call and replace the direct res.json() usage with the safe text-then-JSON parsing/fallback logic so non-JSON error responses (HTML, plain text) are surfaced in the error message.apps/webapp/app/presenters/v3/SpanPresenter.server.ts-841-849 (1)
841-849:⚠️ Potential issue | 🟡 MinorHandle potential NaN from version parsing.
Number(promptVersion)on a non-numeric string returnsNaN, which would cause the Prisma query to fail or return unexpected results. Consider adding validation.🛡️ Proposed fix to validate version number
const version = await this._replica.promptVersion.findUnique({ where: { promptId_version: { promptId: prompt.id, - version: Number(promptVersion), + version: parseInt(promptVersion, 10), }, }, }); - if (!version) return undefined; + if (!version || isNaN(version.version)) return undefined;Alternatively, validate before the query:
+ const versionNum = parseInt(promptVersion, 10); + if (isNaN(versionNum)) return undefined; + const version = await this._replica.promptVersion.findUnique({ where: { promptId_version: { promptId: prompt.id, - version: Number(promptVersion), + version: versionNum, }, }, });🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/presenters/v3/SpanPresenter.server.ts` around lines 841 - 849, The code calls this._replica.promptVersion.findUnique using Number(promptVersion) which can be NaN; before querying, parse and validate promptVersion (e.g., with Number.isFinite / Number.isInteger or parseInt) and handle invalid values by returning undefined or throwing as appropriate; update the logic around the version lookup (the promptVersion input, the Number(...) conversion, and the variable version) so the query only runs with a valid numeric version (and include prompt.id in the check) to avoid passing NaN into Prisma.apps/webapp/app/v3/services/promptService.server.ts-115-116 (1)
115-116:⚠️ Potential issue | 🟡 MinorSame race condition applies to
reactivateOverride.Consider wrapping the
#removeLabeland#addLabelcalls in a transaction here as well for consistency withcreateOverride.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/v3/services/promptService.server.ts` around lines 115 - 116, reactivateOverride currently calls this.#removeLabel(promptId, "override") and this.#addLabel(versionId, "override") separately, causing the same race condition as createOverride; modify reactivateOverride to run both label operations inside a single database transaction (use the same transaction helper or mechanism used by createOverride) so that the remove/add occur atomically, pass the transaction/context into `#removeLabel` and `#addLabel` (or call transactional variants), and ensure errors roll back the transaction and propagate appropriately.apps/webapp/app/v3/services/promptService.server.ts-21-22 (1)
21-22:⚠️ Potential issue | 🟡 MinorPotential race condition between label operations.
#removeLabeland#addLabelare not wrapped in a transaction, so concurrentpromoteVersioncalls could leave the system in an inconsistent state (e.g., no version with "current" label, or multiple versions with it).🔒 Suggested fix: wrap in transaction
- await this.#removeLabel(promptId, "current"); - await this.#addLabel(versionId, "current"); + await this._prisma.$transaction(async (tx) => { + await tx.$executeRaw` + UPDATE "prompt_versions" + SET "labels" = array_remove("labels", 'current') + WHERE "promptId" = ${promptId} AND 'current' = ANY("labels") + `; + await tx.$executeRaw` + UPDATE "prompt_versions" + SET "labels" = array_append("labels", 'current') + WHERE "id" = ${versionId} AND NOT ('current' = ANY("labels")) + `; + });🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/v3/services/promptService.server.ts` around lines 21 - 22, The two label operations (`#removeLabel` and `#addLabel`) can race; wrap them in a single database transaction inside the promoteVersion flow so both run atomically on the same DB session/connection: begin a transaction, call `#removeLabel`(promptId, "current") then `#addLabel`(versionId, "current") using the transactional context, commit on success and rollback on error, and ensure both helper methods accept/propagate the transaction/session (or inline their SQL/ORM calls into the transaction) so concurrent promoteVersion calls cannot leave multiple or zero "current" labels.
🧹 Nitpick comments (16)
apps/webapp/app/components/code/TextEditor.tsx (3)
12-19: Use a type alias forTextEditorPropsLine 12 uses an
interface; this repo standard preferstypealiases in TS/TSX.♻️ Suggested change
-export interface TextEditorProps extends Omit<ReactCodeMirrorProps, "onBlur"> { +export type TextEditorProps = Omit<ReactCodeMirrorProps, "onBlur"> & { defaultValue?: string; readOnly?: boolean; onChange?: (value: string) => void; onUpdate?: (update: ViewUpdate) => void; showCopyButton?: boolean; additionalActions?: React.ReactNode; -} +};As per coding guidelines,
**/*.{ts,tsx}: Use types over interfaces for TypeScript.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/components/code/TextEditor.tsx` around lines 12 - 19, Replace the interface declaration with a type alias: change the exported TextEditorProps interface into an exported type alias that equals Omit<ReactCodeMirrorProps, "onBlur"> & { ... } and keep all existing fields (defaultValue, readOnly, onChange, onUpdate, showCopyButton, additionalActions) unchanged; update any imports/uses still referring to TextEditorProps if needed but the symbol name stays the same (TextEditorProps) and the file TextEditor.tsx should reflect the type alias form.
1-125: Add@crumbstracing markers in this new TSX componentThis file introduces new behavior but has no
@crumbsmarkers. Please add either inline//@Crumbscomments or a `// `#region` `@crumbsblock per repo convention.As per coding guidelines,
**/*.{ts,tsx,js}: Add crumbs as you write code using //@crumbscomments or //#region@crumbsblocks for agentcrumbs debug tracing.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/components/code/TextEditor.tsx` around lines 1 - 125, Add the required `@crumbs` tracing markers to this new TextEditor component by inserting either inline // `@crumbs` comments or a // `#region` `@crumbs` block around the component entry points: place a top-level marker just above the TextEditor function declaration and add markers at key internal points such as inside the useEffect that sets the container (useRef editor/current), the useEffect that syncs defaultValue to view, and the copy callback (copy) to trace user actions and lifecycle; ensure markers reference the function names (TextEditor, copy) and the view/setContainer usage so they are easy to find.
33-46: Use declarative array construction instead of mutationLines 33-46 build
extensionsvia sequentialpush()calls. WhilegetEditorSetup(false)returns a fresh array each invocation (not shared), imperative mutation is less readable and maintainable. Create the array declaratively using the spread operator.♻️ Suggested change
-import { useCallback, useEffect, useRef, useState } from "react"; +import { useCallback, useEffect, useMemo, useRef, useState } from "react"; @@ - const extensions = getEditorSetup(false); - extensions.push(EditorView.lineWrapping); - extensions.push( - lineNumbers({ - formatNumber: (n) => String(n), - }) - ); - extensions.push( - EditorView.theme({ - ".cm-lineNumbers": { - minWidth: "40px", - }, - }) - ); + const extensions = useMemo( + () => [ + ...getEditorSetup(false), + EditorView.lineWrapping, + lineNumbers({ + formatNumber: (n) => String(n), + }), + EditorView.theme({ + ".cm-lineNumbers": { + minWidth: "40px", + }, + }), + ], + [] + );🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/components/code/TextEditor.tsx` around lines 33 - 46, Replace the imperative pushes that mutate the extensions array with a declarative array construction: create a new array by spreading the result of getEditorSetup(false) and appending EditorView.lineWrapping, the lineNumbers(...) call (with formatNumber), and the EditorView.theme(...) object (with ".cm-lineNumbers" minWidth). Update the variable named extensions to be initialized with that single array expression so the code uses a readable immutable pattern around getEditorSetup, EditorView.lineWrapping, lineNumbers, and EditorView.theme.packages/cli-v3/src/entryPoints/managed-index-worker.ts (1)
205-220: Consider consolidating prompt/task schema conversion into one shared helper.The two conversion paths now duplicate the same error-handling and mapping pattern; a generic helper would reduce drift risk.
Also applies to: 222-239
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/cli-v3/src/entryPoints/managed-index-worker.ts` around lines 205 - 220, The convertPromptSchemasToJsonSchemas function duplicates the mapping + try/catch pattern used elsewhere (e.g., the similar task conversion block around lines 222-239); extract a shared helper (e.g., mapWithSchemaConversion) that accepts the items array and a schema lookup function (resourceCatalog.getPromptSchema / getTaskSchema) and performs the schemaToJsonSchema conversion with the same safe try/catch logic, then replace convertPromptSchemasToJsonSchemas and the task conversion with calls to that helper to avoid duplicated error handling and mapping logic.apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.prompts._index/route.tsx (1)
98-100: Unreachable code: empty state already handled earlier.The
TableBlankRowon lines 98-100 will never render becauseprompts.length === 0is already handled with an early return at line 64. This ternary can be simplified to just map the prompts directly.♻️ Suggested simplification
<TableBody> - {prompts.length === 0 ? ( - <TableBlankRow colSpan={7}>No prompts found</TableBlankRow> - ) : ( - prompts.map((prompt) => { + {prompts.map((prompt) => { const path = `${v3PromptsPath(organization, project, environment)}/${prompt.slug}`; const activeVersion = prompt.overrideVersion ?? prompt.currentVersion; const isOverride = !!prompt.overrideVersion; // ... rest of mapping - }) - )} + })} </TableBody>🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/routes/_app.orgs`.$organizationSlug.projects.$projectParam.env.$envParam.prompts._index/route.tsx around lines 98 - 100, The ternary that renders TableBlankRow when prompts.length === 0 is unreachable because an early return handles the empty state; remove the conditional branch and simplify the JSX to directly map over prompts (use the existing prompts array mapping logic) and delete the unreachable TableBlankRow usage; update the render in the route component where prompts is referenced and remove any dead code path that checks prompts.length === 0 (symbols to locate: prompts, TableBlankRow).apps/webapp/app/v3/services/createBackgroundWorker.server.ts (1)
710-711: Move import to the top of the file.The
import { createHash } from "crypto"statement is placed at the end of the file (line 711), which is unconventional and may cause confusion. Node.js built-in imports should be grouped with other imports at the top.♻️ Move import to top
Add at the top with other imports (around line 1-9):
import { createHash } from "crypto";Then remove lines 710-711.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/v3/services/createBackgroundWorker.server.ts` around lines 710 - 711, The file currently contains a stray "import { createHash } from 'crypto'" near the end; move that import up into the existing import block at the top of the file with the other imports (so createHash is imported alongside the top-level imports) and remove the trailing import statement at the bottom (the one currently on lines 710-711) to keep imports grouped and conventional.apps/webapp/app/components/metrics/ProvidersFilter.tsx (1)
18-20: Prefer a type alias for the props shape.This repo standardizes on
typeinstead ofinterfacein TypeScript files. As per coding guidelines, "Use types over interfaces for TypeScript".🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/components/metrics/ProvidersFilter.tsx` around lines 18 - 20, Replace the interface declaration for ProvidersFilterProps with a type alias to follow project conventions: change the declaration "interface ProvidersFilterProps { possibleProviders: string[] }" to a type alias named ProvidersFilterProps with the same shape (possibleProviders: string[]), ensuring any references to ProvidersFilterProps (e.g., in the ProvidersFilter component props) continue to work unchanged.apps/webapp/app/components/metrics/OperationsFilter.tsx (1)
18-20: Prefer a type alias for the props shape.This repo standardizes on
typeinstead ofinterfacein TypeScript files. As per coding guidelines, "Use types over interfaces for TypeScript".🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/components/metrics/OperationsFilter.tsx` around lines 18 - 20, Replace the interface declaration for OperationsFilterProps with a type alias to match repository standards: change the "interface OperationsFilterProps { possibleOperations: string[] }" to a "type OperationsFilterProps = { possibleOperations: string[] }" and ensure any usages of OperationsFilterProps (e.g., in the OperationsFilter component props) continue to reference the same symbol.apps/webapp/app/components/metrics/PromptsFilter.tsx (1)
18-20: Prefer a type alias for the props shape.This repo standardizes on
typeinstead ofinterfacein TypeScript files. As per coding guidelines, "Use types over interfaces for TypeScript".🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/components/metrics/PromptsFilter.tsx` around lines 18 - 20, Replace the interface declaration for PromptsFilterProps with a type alias: change "interface PromptsFilterProps { possiblePrompts: string[] }" to "type PromptsFilterProps = { possiblePrompts: string[] }"; update any usages or exports of PromptsFilterProps as needed (no other logic changes).apps/webapp/app/components/runs/v3/ai/AISpanDetails.tsx (1)
296-311: Consider extracting shared helpers to reduce duplication.
PromptMetricRowandtryPrettyJsonare duplicated inAIToolCallSpanDetails.tsx. Consider extracting these to a shared module (e.g.,~/components/runs/v3/ai/shared.tsx) if this pattern continues to expand.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/components/runs/v3/ai/AISpanDetails.tsx` around lines 296 - 311, Extract the duplicated PromptMetricRow component and tryPrettyJson function into a shared module (e.g., create a new file like shared.tsx) and replace the local definitions in AISpanDetails.tsx and AIToolCallSpanDetails.tsx with imports from that module; specifically move the PromptMetricRow function and the tryPrettyJson function into the shared file, export them, update both files to import { PromptMetricRow, tryPrettyJson } from the new shared module, and run a quick typecheck to ensure props and return types align.apps/webapp/app/routes/api.v1.prompts.$slug.ts (1)
152-174: Template compiler has limited Mustache compatibility.The
compileTemplatefunction handles basic{{key}}interpolation and simple{{#key}}...{{/key}}blocks, but:
- Nested blocks (
{{#outer}}{{#inner}}...{{/inner}}{{/outer}}) won't work correctly due to greedy regex- Inverted sections (
{{^key}}) are not supported- Comments (
{{! comment }}) are not strippedIf this is intentional for a simplified template syntax, consider documenting the supported syntax. Otherwise, consider using a lightweight template library.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/routes/api.v1.prompts`.$slug.ts around lines 152 - 174, The compileTemplate function currently uses greedy regexes which break nested sections (e.g., {{`#outer`}}{{`#inner`}}...{{/inner}}{{/outer}}), and it does not handle inverted sections ({{^key}}) or strip comments ({{! ... }}); either document these limitations in the function's JSDoc and README as intentional, or replace the implementation with a proper Mustache-compatible renderer: remove the ad-hoc regex logic in compileTemplate and call a lightweight library (e.g., mustache.render or hogan.js) to get correct handling of nested blocks, inverted sections, and comments, or implement a small recursive parser that matches opening/closing tags to support nesting and add support for ^-sections and comment stripping before interpolation.internal-packages/database/prisma/schema.prisma (1)
588-588: Consider documenting expectedsourcevalues.The
sourcefield is a String with comment noting expected values. Consider adding a more detailed comment or using a Prisma@defaultvalue to make the expected values clearer for future maintainers.source String // Expected: "code" | "dashboard" | "api"🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@internal-packages/database/prisma/schema.prisma` at line 588, The schema's source field currently is a plain String (source) with an inline comment; change it to a stronger, self-documenting type or add a clearer default/comment: either define a Prisma enum (e.g., enum Source { deploy dashboard api }) and replace source String with source Source, or add a more explicit inline comment and a sensible `@default` (e.g., `@default`("dashboard")) to schema.prisma so the expected values ("deploy" | "dashboard" | "api") are enforced/visible; update any code that writes/reads this field to use the enum values if you choose the enum route (look for usages of source in migrations, models, and repository functions).packages/trigger-sdk/src/v3/prompt.ts (2)
56-67: Template conditional section regex may not handle nested conditionals.The regex
/\{\{#(\w+)\}\}([\s\S]*?)\{\{\/\1\}\}/guses non-greedy matching which works for simple cases, but nested conditionals like{{#a}}{{#b}}...{{/b}}{{/a}}would be incorrectly parsed due to the first{{/being matched with the outer{{#a}}.If nested conditionals are not a supported feature, consider adding a note in the docstring. Otherwise, a recursive or stack-based parser would be needed.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/trigger-sdk/src/v3/prompt.ts` around lines 56 - 67, The current conditional parser using template.replace with the regex /\{\{#(\w+)\}\}([\s\S]*?)\{\{\/\1\}\}/g (assigned to result) does not support nested conditionals (e.g., {{`#a`}}{{`#b`}}...{{/b}}{{/a}}); either explicitly document in the prompt module's function comment that nested conditionals are not supported, or replace the regex-based approach with a simple recursive/stack-based parser: parse the template char-by-char, push opening tags ({{`#key`}}) onto a stack, build node/content trees, and evaluate nodes using variables to correctly handle nested {{#...}}{{/...}} blocks—update the function that currently performs the replace to use this parser and keep existing variable interpolation logic for inner content.
160-208: API resolution lacks explicit error handling.If
apiClient.resolvePromptthrows or returns an error response, the promise rejects without a user-friendly message. Consider wrapping in try/catch to provide context about prompt resolution failures.💡 Suggested improvement
if (ctx && apiClient) { + try { const response = await apiClient.resolvePrompt( options.id, // ... existing code ... ); // ... existing return logic ... + } catch (error) { + throw new Error( + `Failed to resolve prompt "${options.id}": ${error instanceof Error ? error.message : String(error)}` + ); + } }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/trigger-sdk/src/v3/prompt.ts` around lines 160 - 208, Wrap the apiClient.resolvePrompt(...) call inside a try/catch in the same block where ctx and apiClient are checked (the prompt resolution path that currently invokes apiClient.resolvePrompt with tracer/name "prompt.resolve()"); on catch, create a clear, contextual error message that includes the prompt id (options.id) and any relevant resolveOptions (label/version) and either log it via the existing logger/tracer or throw a new Error that wraps the original error so callers get a user-friendly message; ensure you preserve and rethrow the original error as the cause if available so stack traces remain useful.apps/webapp/app/presenters/v3/PromptPresenter.server.ts (2)
6-32: Consider consolidating GenerationRow definitions.The
GenerationRowSchema(zod) andGenerationRow(type) have slightly different field names (started_atvsstart_time). While the transformation at line 334 handles this, having two similar definitions could lead to drift.Consider deriving the type from the schema:
type GenerationRowRaw = z.infer<typeof GenerationRowSchema>;🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/presenters/v3/PromptPresenter.server.ts` around lines 6 - 32, The GenerationRowSchema and GenerationRow types are duplicated and have a mismatched field name (started_at vs start_time); derive a single source of truth by using z.infer<typeof GenerationRowSchema> (e.g., type GenerationRowRaw = z.infer<typeof GenerationRowSchema>) and then export/rename a cleaned type for callers (GenerationRow) that matches the rest of the code, and update the transformation that converts started_at to start_time to return the derived/renamed type so you no longer maintain two divergent definitions.
128-141: Timezone mismatch risk in sparkline bucket alignment.The bucket key generation uses
toISOString().slice(0, 13)which produces UTC timestamps, but ClickHouse'stoStartOfHour(start_time)may use the server's timezone setting. If they don't align, counts will be attributed to wrong buckets.Consider explicitly using UTC in the ClickHouse query:
💡 Suggested fix
query: `SELECT prompt_slug, - toStartOfHour(start_time) AS bucket, + formatDateTime(toStartOfHour(start_time), '%Y-%m-%d %H:00:00', 'UTC') AS bucket, count() AS cntAnd update the JavaScript to match:
bucketKeys.push( - h.toISOString().slice(0, 13).replace("T", " ") + ":00:00" + h.toISOString().slice(0, 13).replace("T", " ") + ":00:00" // Already UTC );🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/webapp/app/presenters/v3/PromptPresenter.server.ts` around lines 128 - 141, The bucket generation uses local-hour math then toISOString (UTC), which can misalign with ClickHouse; switch to pure-UTC calculations and make ClickHouse use UTC for hour bucketing. Concretely: in PromptPresenter.server.ts change the startHour computation and per-hour loop to compute times in milliseconds from Date.now() (or use getUTCHours/getUTC*), e.g. derive startUtcMs = Date.now() - 23*3600*1000 and build each bucket with new Date(startUtcMs + i*3600*1000) so toISOString() yields the intended UTC hour strings; and update the ClickHouse query to call toStartOfHour(start_time, 'UTC') (or the equivalent timezone param) so both JS bucketKeys and ClickHouse grouping use UTC consistently.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: a29af65c-2bdd-4a1f-b68d-9bc64a956682
⛔ Files ignored due to path filters (3)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yamlreferences/hello-world/package.jsonis excluded by!references/**references/hello-world/src/trigger/prompts.tsis excluded by!references/**
📒 Files selected for processing (74)
apps/webapp/app/components/BlankStatePanels.tsxapps/webapp/app/components/code/TSQLResultsTable.tsxapps/webapp/app/components/code/TextEditor.tsxapps/webapp/app/components/metrics/OperationsFilter.tsxapps/webapp/app/components/metrics/PromptsFilter.tsxapps/webapp/app/components/metrics/ProvidersFilter.tsxapps/webapp/app/components/metrics/QueryWidget.tsxapps/webapp/app/components/navigation/SideMenu.tsxapps/webapp/app/components/primitives/Resizable.tsxapps/webapp/app/components/runs/v3/PromptSpanDetails.tsxapps/webapp/app/components/runs/v3/SpanTitle.tsxapps/webapp/app/components/runs/v3/ai/AIChatMessages.tsxapps/webapp/app/components/runs/v3/ai/AIEmbedSpanDetails.tsxapps/webapp/app/components/runs/v3/ai/AIModelSummary.tsxapps/webapp/app/components/runs/v3/ai/AISpanDetails.tsxapps/webapp/app/components/runs/v3/ai/AIToolCallSpanDetails.tsxapps/webapp/app/components/runs/v3/ai/extractAISpanData.tsapps/webapp/app/components/runs/v3/ai/extractAISummarySpanData.tsapps/webapp/app/components/runs/v3/ai/index.tsapps/webapp/app/components/runs/v3/ai/types.tsapps/webapp/app/presenters/v3/BuiltInDashboards.server.tsapps/webapp/app/presenters/v3/MetricDashboardPresenter.server.tsapps/webapp/app/presenters/v3/PromptPresenter.server.tsapps/webapp/app/presenters/v3/SpanPresenter.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.dashboards.$dashboardKey/route.tsxapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.prompts.$promptSlug/route.tsxapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.prompts._index/route.tsxapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam/route.tsxapps/webapp/app/routes/api.v1.prompts.$slug.override.reactivate.tsapps/webapp/app/routes/api.v1.prompts.$slug.override.tsapps/webapp/app/routes/api.v1.prompts.$slug.promote.tsapps/webapp/app/routes/api.v1.prompts.$slug.tsapps/webapp/app/routes/api.v1.prompts.$slug.versions.tsapps/webapp/app/routes/api.v1.prompts._index.tsapps/webapp/app/routes/resources.metric.tsxapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.prompts.$promptSlug.generations.tsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsxapps/webapp/app/services/authorization.server.tsapps/webapp/app/services/queryService.server.tsapps/webapp/app/utils/pathBuilder.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.tsapps/webapp/app/v3/eventRepository/eventRepository.types.tsapps/webapp/app/v3/querySchemas.tsapps/webapp/app/v3/services/createBackgroundWorker.server.tsapps/webapp/app/v3/services/promptService.server.tsapps/webapp/app/v3/utils/enrichCreatableEvents.server.tsinternal-packages/clickhouse/schema/025_add_prompt_columns_to_llm_metrics_v1.sqlinternal-packages/clickhouse/src/llmMetrics.tsinternal-packages/database/prisma/migrations/20260317221618_add_prompt_management/migration.sqlinternal-packages/database/prisma/migrations/20260318114244_add_prompt_friendly_id/migration.sqlinternal-packages/database/prisma/schema.prismainternal-packages/llm-pricing/src/default-model-prices.jsoninternal-packages/llm-pricing/src/defaultPrices.tspackage.jsonpackages/cli-v3/src/dev/devSupervisor.tspackages/cli-v3/src/entryPoints/dev-index-worker.tspackages/cli-v3/src/entryPoints/managed-index-worker.tspackages/cli-v3/src/mcp/config.tspackages/cli-v3/src/mcp/tools.tspackages/cli-v3/src/mcp/tools/prompts.tspackages/core/src/v3/apiClient/index.tspackages/core/src/v3/resource-catalog/catalog.tspackages/core/src/v3/resource-catalog/index.tspackages/core/src/v3/resource-catalog/noopResourceCatalog.tspackages/core/src/v3/resource-catalog/standardResourceCatalog.tspackages/core/src/v3/schemas/api.tspackages/core/src/v3/schemas/build.tspackages/core/src/v3/schemas/resources.tspackages/core/src/v3/schemas/schemas.tspackages/core/src/v3/types/tasks.tspackages/trigger-sdk/src/v3/index.tspackages/trigger-sdk/src/v3/prompt.tspackages/trigger-sdk/src/v3/prompts.tspatches/@window-splitter__state@0.4.1.patch
| const copy = useCallback(() => { | ||
| if (view === undefined) return; | ||
| navigator.clipboard.writeText(view.state.doc.toString()); | ||
| setCopied(true); | ||
| setTimeout(() => setCopied(false), 1500); | ||
| }, [view]); |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Does navigator.clipboard.writeText() return a Promise that can reject, and in which cases (permissions/insecure context/user gesture)?
💡 Result:
Yes, navigator.clipboard.writeText returns a Promise that can reject with a NotAllowedError DOMException. It rejects in these cases: - Insecure context (non-HTTPS, non-localhost): navigator.clipboard is undefined, so calling writeText throws a TypeError (property undefined). - Secure context but permission denied: e.g., clipboard-write permission not granted (Chromium), or lacking transient user activation (user gesture like click; required by most browsers including Safari/Firefox strictly, Chromium with conditions). - Not in active tab or lacking document focus (browser-specific, e.g., Chromium requires active tab). - Permissions Policy disallows clipboard-write (e.g., in iframes without proper header). Always wrap in try-catch or .catch as shown in MDN examples.
Citations:
- 1: https://developer.mozilla.org/en-US/docs/Web/API/Clipboard/writeText
- 2: https://developer.mozilla.org/en-US/docs/Web/API/Clipboard
- 3: https://web.dev/articles/async-clipboard
- 4: https://jsguides.dev/tutorials/browser-clipboard/
- 5: https://developer.mozilla.org/docs/Web/API/Clipboard_API
- 6: https://w3c.github.io/clipboard-apis/
- 7: https://stackoverflow.com/questions/51805395/navigator-clipboard-is-undefined
- 8: Issue with "Clipboard.writeText()": doesn't work in HTTP protocol mdn/content#10741
- 9: https://bugzilla.mozilla.org/show_bug.cgi?id=1554855
- 10: User gesture requirement for Clipboard API access w3c/clipboard-apis#52
- 11: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Interact_with_the_clipboard
- 12: Interoperability issue:
navigator.clipboard.write()andnavigator.clipboard.writeText()user gesture requirement w3c/clipboard-apis#182
🏁 Script executed:
# Check if the file exists and examine the exact code at lines 81-86
if [ -f "apps/webapp/app/components/code/TextEditor.tsx" ]; then
echo "File found. Showing lines 78-90 for context:"
sed -n '78,90p' "apps/webapp/app/components/code/TextEditor.tsx"
else
echo "File not found. Searching for TextEditor.tsx..."
find . -name "TextEditor.tsx" -type f 2>/dev/null | head -5
fiRepository: triggerdotdev/trigger.dev
Length of output: 434
Handle clipboard write failures before showing success
The navigator.clipboard.writeText() call on line 83 returns a Promise that can reject due to permission denial, insecure context, lacking user activation, or permissions policy violations. Without error handling, a rejection surfaces as an unhandled promise rejection and the "copied" UI state shows incorrectly.
Suggested change
- const copy = useCallback(() => {
+ const copy = useCallback(async () => {
if (view === undefined) return;
- navigator.clipboard.writeText(view.state.doc.toString());
- setCopied(true);
- setTimeout(() => setCopied(false), 1500);
+ try {
+ await navigator.clipboard.writeText(view.state.doc.toString());
+ setCopied(true);
+ setTimeout(() => setCopied(false), 1500);
+ } catch {
+ setCopied(false);
+ }
}, [view]);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const copy = useCallback(() => { | |
| if (view === undefined) return; | |
| navigator.clipboard.writeText(view.state.doc.toString()); | |
| setCopied(true); | |
| setTimeout(() => setCopied(false), 1500); | |
| }, [view]); | |
| const copy = useCallback(async () => { | |
| if (view === undefined) return; | |
| try { | |
| await navigator.clipboard.writeText(view.state.doc.toString()); | |
| setCopied(true); | |
| setTimeout(() => setCopied(false), 1500); | |
| } catch { | |
| setCopied(false); | |
| } | |
| }, [view]); |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@apps/webapp/app/components/code/TextEditor.tsx` around lines 81 - 86, The
copy function in TextEditor currently calls
navigator.clipboard.writeText(view.state.doc.toString()) without awaiting or
handling failures, causing unhandled promise rejections and incorrect setCopied
UI state; update the copy callback (the useCallback named copy) to await
navigator.clipboard.writeText(...) or chain .then/.catch, only call
setCopied(true) on success, call setCopied(false) or leave unchanged on failure,
and log or surface the error (e.g., via console.error or an existing logger) so
permission/context failures are handled gracefully.
| function OperationsDropdown({ | ||
| trigger, | ||
| clearSearchValue, | ||
| searchValue, | ||
| onClose, | ||
| possibleOperations, | ||
| }: { | ||
| trigger: ReactNode; | ||
| clearSearchValue: () => void; | ||
| searchValue: string; | ||
| onClose?: () => void; | ||
| possibleOperations: string[]; | ||
| }) { | ||
| const { values, replace } = useSearchParams(); | ||
|
|
||
| const handleChange = (values: string[]) => { | ||
| clearSearchValue(); | ||
| replace({ operations: values }); | ||
| }; | ||
|
|
||
| const filtered = useMemo(() => { | ||
| const q = searchValue.toLowerCase(); | ||
| return possibleOperations.filter( | ||
| (op) => op.toLowerCase().includes(q) || formatOperation(op).toLowerCase().includes(q) | ||
| ); | ||
| }, [searchValue, possibleOperations]); |
There was a problem hiding this comment.
Wire the combobox back to the search state.
filtered only reacts to searchValue, but OperationsDropdown only receives clearSearchValue and ComboBox only gets the current value. Typing here won't update the rendered list, so the operation search is effectively broken.
Also applies to: 113-125
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@apps/webapp/app/components/metrics/OperationsFilter.tsx` around lines 85 -
110, The filtering is only tied to searchValue while the ComboBox input isn't
updating that state; update OperationsDropdown so the ComboBox input drives the
search used by filtered: either propagate ComboBox input changes into the search
param (useSearchParams values/replace) or maintain a local inputState in
OperationsDropdown and include it in the useMemo dependency list; ensure
ComboBox receives the current input value and an onInputChange that updates that
state (so filtered recalculates), and keep handleChange to update the selected
operations via replace({ operations: values }) and clearSearchValue as before;
adjust the logic around filtered, ComboBox props, and any references to
searchValue in OperationsDropdown/handleChange accordingly.
| function PromptsDropdown({ | ||
| trigger, | ||
| clearSearchValue, | ||
| searchValue, | ||
| onClose, | ||
| possiblePrompts, | ||
| }: { | ||
| trigger: ReactNode; | ||
| clearSearchValue: () => void; | ||
| searchValue: string; | ||
| onClose?: () => void; | ||
| possiblePrompts: string[]; | ||
| }) { | ||
| const { values, replace } = useSearchParams(); | ||
|
|
||
| const handleChange = (values: string[]) => { | ||
| clearSearchValue(); | ||
| replace({ prompts: values }); | ||
| }; | ||
|
|
||
| const filtered = useMemo(() => { | ||
| return possiblePrompts.filter((p) => { | ||
| return p.toLowerCase().includes(searchValue.toLowerCase()); | ||
| }); | ||
| }, [searchValue, possiblePrompts]); |
There was a problem hiding this comment.
Wire the combobox back to the search state.
filtered only reacts to searchValue, but PromptsDropdown only receives clearSearchValue and ComboBox only gets the current value. Typing here won't update the rendered list, so the prompt search is effectively broken.
Also applies to: 101-113
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@apps/webapp/app/components/metrics/PromptsFilter.tsx` around lines 74 - 98,
PromptsDropdown's filtered list only depends on the local searchValue, but the
component reads/writes the persistent search state via useSearchParams (values,
replace) and the ComboBox is controlled by that state, so typing doesn't update
the rendered list; fix by reading the current prompts/search term from values
(values.prompts or a dedicated values.search key) and include that in the
filtered useMemo dependencies, ensure handleChange calls replace with the
updated prompts array/string, and make the ComboBox controlled by the same
search param (pass the parsed values.prompts as its value and wire its onChange
to handleChange/replace while still calling clearSearchValue where appropriate)
so filtered, ComboBox value, and URL/search params stay in sync (adjust parsing
if values.prompts is a string vs array).
| function ProvidersDropdown({ | ||
| trigger, | ||
| clearSearchValue, | ||
| searchValue, | ||
| onClose, | ||
| possibleProviders, | ||
| }: { | ||
| trigger: ReactNode; | ||
| clearSearchValue: () => void; | ||
| searchValue: string; | ||
| onClose?: () => void; | ||
| possibleProviders: string[]; | ||
| }) { | ||
| const { values, replace } = useSearchParams(); | ||
|
|
||
| const handleChange = (values: string[]) => { | ||
| clearSearchValue(); | ||
| replace({ providers: values }); | ||
| }; | ||
|
|
||
| const filtered = useMemo(() => { | ||
| return possibleProviders.filter((p) => p.toLowerCase().includes(searchValue.toLowerCase())); | ||
| }, [searchValue, possibleProviders]); |
There was a problem hiding this comment.
Wire the combobox back to the search state.
filtered only reacts to searchValue, but ProvidersDropdown only receives clearSearchValue and ComboBox only gets the current value. Typing here won't update the rendered list, so the provider search is effectively broken.
Also applies to: 99-111
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@apps/webapp/app/components/metrics/ProvidersFilter.tsx` around lines 74 - 96,
ProvidersDropdown’s filtered list is only using the searchValue prop so typing
in the ComboBox doesn’t update the rendered list; change filtered to derive its
input from the search state returned by useSearchParams (the values object)
instead of — or in addition to — the searchValue prop, and wire the ComboBox
input handlers to update that search param via replace (and call
clearSearchValue where appropriate). Specifically, update the useMemo that
computes filtered to depend on values (or the exact search param key you use,
e.g. values.providerSearch or values.providers) and possibleProviders, and
ensure the ComboBox receives the controlled input from values and calls a
handler that calls replace(...) so typing updates values and recomputes
filtered; keep handleChange for committing provider selections.
| <SideMenuSection | ||
| title="AI" | ||
| isSideMenuCollapsed={isCollapsed} | ||
| itemSpacingClassName="space-y-0" | ||
| initialCollapsed={getSectionCollapsed( | ||
| user.dashboardPreferences.sideMenu, | ||
| "ai" | ||
| )} | ||
| onCollapseToggle={handleSectionToggle("ai")} | ||
| > | ||
| <SideMenuItem | ||
| name="Prompts" | ||
| icon={DocumentTextIcon} | ||
| activeIconColor="text-purple-500" | ||
| inactiveIconColor="text-purple-500" | ||
| to={v3PromptsPath(organization, project, environment)} | ||
| data-action="prompts" | ||
| isCollapsed={isCollapsed} | ||
| /> | ||
| <SideMenuItem | ||
| name="AI Metrics" | ||
| icon={SparklesIcon} | ||
| activeIconColor="text-purple-500" | ||
| inactiveIconColor="text-purple-500" | ||
| to={v3BuiltInDashboardPath(organization, project, environment, "llm")} | ||
| data-action="ai-metrics" | ||
| isCollapsed={isCollapsed} | ||
| /> | ||
| </SideMenuSection> | ||
|
|
There was a problem hiding this comment.
Gate “AI Metrics” behind the same query-access check used for dashboards.
Right now AI Metrics is always rendered, which can expose a nav path users may not be authorized to access.
Proposed fix
<SideMenuItem
name="Prompts"
icon={DocumentTextIcon}
activeIconColor="text-purple-500"
inactiveIconColor="text-purple-500"
to={v3PromptsPath(organization, project, environment)}
data-action="prompts"
isCollapsed={isCollapsed}
/>
- <SideMenuItem
- name="AI Metrics"
- icon={SparklesIcon}
- activeIconColor="text-purple-500"
- inactiveIconColor="text-purple-500"
- to={v3BuiltInDashboardPath(organization, project, environment, "llm")}
- data-action="ai-metrics"
- isCollapsed={isCollapsed}
- />
+ {(user.admin || user.isImpersonating || featureFlags.hasQueryAccess) && (
+ <SideMenuItem
+ name="AI Metrics"
+ icon={SparklesIcon}
+ activeIconColor="text-purple-500"
+ inactiveIconColor="text-purple-500"
+ to={v3BuiltInDashboardPath(organization, project, environment, "llm")}
+ data-action="ai-metrics"
+ isCollapsed={isCollapsed}
+ />
+ )}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@apps/webapp/app/components/navigation/SideMenu.tsx` around lines 460 - 489,
The "AI Metrics" SideMenuItem is always rendered and should be conditionally
shown using the same query-access check used for dashboards; wrap the
SideMenuItem that links to v3BuiltInDashboardPath(...) in that existing
permission check (the same helper used when rendering the dashboards nav), so
only render the SideMenuItem when the check passes—leave the "Prompts" item and
the surrounding SideMenuSection, getSectionCollapsed and handleSectionToggle
logic unchanged.
| if (method === "POST") { | ||
| const rawBody = await request.json(); | ||
| const parsed = CreateBody.safeParse(rawBody); | ||
| if (!parsed.success) { | ||
| return apiCors( | ||
| request, | ||
| json({ error: "Invalid request body", issues: parsed.error.issues }, { status: 400 }) | ||
| ); |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cat -n apps/webapp/app/routes/api.v1.prompts.\$slug.override.tsRepository: triggerdotdev/trigger.dev
Length of output: 4353
🏁 Script executed:
# Check if there are any error boundaries or middleware in this route
head -100 apps/webapp/app/routes/api.v1.prompts.\$slug.override.ts | tail -50Repository: triggerdotdev/trigger.dev
Length of output: 1481
🏁 Script executed:
# Look for any error handling patterns in the file
rg -A 5 -B 5 "request\.json\|try|catch" apps/webapp/app/routes/api.v1.prompts.\$slug.override.tsRepository: triggerdotdev/trigger.dev
Length of output: 341
Wrap request.json() in try-catch to handle malformed JSON gracefully.
On lines 64 and 84, request.json() throws SyntaxError for invalid JSON and bypasses the validation checks below, resulting in unhandled 500 errors. Catch these errors and return a CORS-wrapped 400 response instead.
Also fix line 115: the Allow header should include PATCH since the handler accepts it (line 83).
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@apps/webapp/app/routes/api.v1.prompts`.$slug.override.ts around lines 63 -
70, The handler currently calls request.json() directly (e.g., where
CreateBody.safeParse is used) which throws on malformed JSON; wrap each
request.json() call used for POST and PATCH (the bodies parsed by
CreateBody.safeParse and the corresponding UpdateBody.safeParse) in try-catch
and on catch return apiCors(request, json({ error: "Invalid JSON" }, { status:
400 })); also update the response that sets the Allow header (the branch
returning 405) to include "PATCH" alongside the other methods so the Allow
header correctly reflects accepted methods.
| const url = new URL(request.url); | ||
| const versions = url.searchParams.getAll("versions").filter(Boolean).map(Number).filter((n) => !isNaN(n)); | ||
| const period = url.searchParams.get("period") ?? "7d"; | ||
| const fromTime = url.searchParams.get("from"); | ||
| const toTime = url.searchParams.get("to"); | ||
| const cursorParam = url.searchParams.get("cursor") ?? undefined; | ||
|
|
||
| const periodMs = parsePeriodToMs(period); | ||
| const startTime = fromTime ? new Date(fromTime) : new Date(Date.now() - periodMs); | ||
| const endTime = toTime ? new Date(toTime) : new Date(); | ||
|
|
||
| const models = url.searchParams.getAll("models").filter(Boolean); | ||
| const operations = url.searchParams.getAll("operations").filter(Boolean); | ||
| const providers = url.searchParams.getAll("providers").filter(Boolean); |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, check the file exists and get its content
cat -n apps/webapp/app/routes/resources.orgs.\$organizationSlug.projects.\$projectParam.env.\$envParam.prompts.\$promptSlug.generations.ts | head -100Repository: triggerdotdev/trigger.dev
Length of output: 3777
🏁 Script executed:
# Check how listGenerations uses these parameters
fd -t f "PromptPresenter" --exec grep -l "listGenerations" {} \;Repository: triggerdotdev/trigger.dev
Length of output: 126
🏁 Script executed:
# Check the PromptPresenter implementation
cat -n ./apps/webapp/app/presenters/v3/PromptPresenter.server.ts | head -150Repository: triggerdotdev/trigger.dev
Length of output: 5574
🏁 Script executed:
# Continue reading to find listGenerations method
cat -n ./apps/webapp/app/presenters/v3/PromptPresenter.server.ts | sed -n '150,350p'Repository: triggerdotdev/trigger.dev
Length of output: 7809
🏁 Script executed:
# Verify Invalid Date behavior by checking if toISOString is called
grep -A 5 "toISOString" ./apps/webapp/app/presenters/v3/PromptPresenter.server.ts | head -10Repository: triggerdotdev/trigger.dev
Length of output: 376
🏁 Script executed:
# Check if there are any zod schemas used in the loader for query validation
grep -B5 -A10 "loader = async" ./apps/webapp/app/routes/resources.orgs.\$organizationSlug.projects.\$projectParam.env.\$envParam.prompts.\$promptSlug.generations.tsRepository: triggerdotdev/trigger.dev
Length of output: 715
🏁 Script executed:
# Verify what happens when Invalid Date is used - check if there are any safeguards
grep -rn "Invalid Date\|toISOString\|RangeError" ./apps/webapp/app/routes/resources.orgs.\$organizationSlug.projects.\$projectParam.env.\$envParam.prompts.\$promptSlug.generations.tsRepository: triggerdotdev/trigger.dev
Length of output: 51
🏁 Script executed:
# Double-check the date parsing and usage one more time to be thorough
sed -n '52,80p' ./apps/webapp/app/routes/resources.orgs.\$organizationSlug.projects.\$projectParam.env.\$envParam.prompts.\$promptSlug.generations.tsRepository: triggerdotdev/trigger.dev
Length of output: 1362
Use zod to validate query parameters before constructing the time window.
Query strings for from, to, versions, period, models, operations, and providers are parsed without validation. new Date(fromTime) and new Date(toTime) can produce Invalid Date objects, which will throw a RangeError when .toISOString() is called at line 300 in the presenter. Additionally, Number() accepts decimals and Infinity, not just valid prompt version integers. These invalid inputs should fail fast with a 400 response, not propagate into listGenerations(). Per coding guidelines, "Use zod for validation in packages/core and apps/webapp".
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@apps/webapp/app/routes/resources.orgs`.$organizationSlug.projects.$projectParam.env.$envParam.prompts.$promptSlug.generations.ts
around lines 52 - 65, The query parameters (from/to, versions, period, models,
operations, providers) are parsed without validation which can yield Invalid
Date or bad numbers; add a zod schema at the top of the request handler in
generations.ts to validate and coerce these query params (validate period with
parsePeriodToMs-compatible values, parse versions as integers >0, ensure from/to
are valid ISO datetimes, and models/operations/providers are arrays of non-empty
strings), return a 400 response on schema failure, and only then construct
startTime/endTime (using new Date(...) on validated values) and call
listGenerations; reference parsePeriodToMs, the URL search params (versions,
period, from/to, models, operations, providers), and listGenerations when wiring
the validated values through.
| @@ -0,0 +1,5 @@ | |||
| -- AlterTable | |||
| ALTER TABLE "public"."prompts" ADD COLUMN "friendly_id" TEXT NOT NULL; | |||
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, let's read the complete migration file
cat internal-packages/database/prisma/migrations/20260318114244_add_prompt_friendly_id/migration.sqlRepository: triggerdotdev/trigger.dev
Length of output: 253
🏁 Script executed:
# Check if there are any related migrations that might create indexes on this column
fd "migration.sql" internal-packages/database/prisma/migrations | xargs grep -l "friendly_id" 2>/dev/nullRepository: triggerdotdev/trigger.dev
Length of output: 262
🏁 Script executed:
# Let's also check if there's any context about the prompts table structure
fd "migration.sql" internal-packages/database/prisma/migrations | xargs grep -l "CREATE TABLE.*prompts" 2>/dev/null | head -5Repository: triggerdotdev/trigger.dev
Length of output: 164
Separate this migration into two files: (1) add friendly_id column with staged rollout, (2) create unique index in separate migration with CONCURRENTLY.
Adding NOT NULL directly to an existing table without a staged migration (nullable column → backfill → add constraint) will fail on environments with existing data. Additionally, the unique index must be created in a separate migration file using CREATE UNIQUE INDEX CONCURRENTLY IF NOT EXISTS to avoid table locks in production.
Current migration structure (problematic)
ALTER TABLE "public"."prompts" ADD COLUMN "friendly_id" TEXT NOT NULL;
CREATE UNIQUE INDEX "prompts_friendly_id_key" ON "public"."prompts"("friendly_id");🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@internal-packages/database/prisma/migrations/20260318114244_add_prompt_friendly_id/migration.sql`
at line 2, Split the single migration into two migrations: (1) change the ALTER
TABLE "prompts" ADD COLUMN "friendly_id" TEXT NOT NULL into a migration that
adds the column as nullable (ALTER TABLE ... ADD COLUMN "friendly_id" TEXT), run
a backfill step to populate friendly_id for existing rows, and then in a
follow-up migration add the NOT NULL constraint (ALTER TABLE ... ALTER COLUMN
"friendly_id" SET NOT NULL); (2) move the unique index creation into a separate
migration and create it using a non-blocking statement such as CREATE UNIQUE
INDEX CONCURRENTLY IF NOT EXISTS "prompts_friendly_id_key" ON
"public"."prompts"("friendly_id") so the index is built without locking the
table. Ensure the migrations reference the column name friendly_id, the table
prompts, and the index name prompts_friendly_id_key so they apply cleanly in
order.
| ALTER TABLE "public"."prompts" ADD COLUMN "friendly_id" TEXT NOT NULL; | ||
|
|
||
| -- CreateIndex | ||
| CREATE UNIQUE INDEX "prompts_friendly_id_key" ON "public"."prompts"("friendly_id"); |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify prompts-table migration history and current index statement usage
fd migration.sql internal-packages/database/prisma/migrations -x rg -n \
'CREATE TABLE "public"\."prompts"|ALTER TABLE "public"\."prompts"|prompts_friendly_id_key|CREATE UNIQUE INDEX'Repository: triggerdotdev/trigger.dev
Length of output: 23196
🏁 Script executed:
cat -n internal-packages/database/prisma/migrations/20260318114244_add_prompt_friendly_id/migration.sqlRepository: triggerdotdev/trigger.dev
Length of output: 288
Split the index creation into a separate migration file using CONCURRENTLY.
Line 5 creates a unique index on an existing table in the same migration as the column addition. Per coding guidelines, this should be split into a separate migration file with CREATE UNIQUE INDEX CONCURRENTLY IF NOT EXISTS "prompts_friendly_id_key" ON "public"."prompts"("friendly_id"); to avoid table locks in production.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@internal-packages/database/prisma/migrations/20260318114244_add_prompt_friendly_id/migration.sql`
at line 5, Move the unique index creation for "prompts_friendly_id_key" out of
this migration and into a new, separate migration that runs after the column
addition; in that new migration use CREATE UNIQUE INDEX CONCURRENTLY IF NOT
EXISTS "prompts_friendly_id_key" ON "public"."prompts"("friendly_id") so the
index is created without taking an exclusive lock on the existing "prompts"
table (reference the index name prompts_friendly_id_key, table "prompts" and
column "friendly_id" when creating the new migration).
| { | ||
| "id": "4bf01a9f-663f-4302-a05c-b2b42c5348e3", | ||
| "modelName": "gpt-5.4-mini", | ||
| "matchPattern": "(?i)^(openai\\/)?(gpt-5.4-mini)$", | ||
| "createdAt": "2026-03-18T00:00:00.000Z", | ||
| "updatedAt": "2026-03-18T00:00:00.000Z", | ||
| "tokenizerConfig": { | ||
| "tokensPerName": 1, | ||
| "tokenizerModel": "gpt-4", | ||
| "tokensPerMessage": 3 | ||
| }, | ||
| "tokenizerId": "openai", | ||
| "pricingTiers": [ | ||
| { | ||
| "id": "4bf01a9f-663f-4302-a05c-b2b42c5348e3_tier_default", | ||
| "name": "Standard", | ||
| "isDefault": true, | ||
| "priority": 0, | ||
| "conditions": [], | ||
| "prices": { | ||
| "input": 0.75e-6, | ||
| "input_cached_tokens": 0.075e-6, | ||
| "input_cache_read": 0.075e-6, | ||
| "output": 4.5e-6 | ||
| } | ||
| } | ||
| ] | ||
| }, | ||
| { | ||
| "id": "3d8027df-7491-442e-9770-1363ab2452fc", | ||
| "modelName": "gpt-5.4-mini-2026-03-17", | ||
| "matchPattern": "(?i)^(openai\\/)?(gpt-5.4-mini-2026-03-17)$", | ||
| "createdAt": "2026-03-18T00:00:00.000Z", | ||
| "updatedAt": "2026-03-18T00:00:00.000Z", | ||
| "tokenizerConfig": { | ||
| "tokensPerName": 1, | ||
| "tokenizerModel": "gpt-4", | ||
| "tokensPerMessage": 3 | ||
| }, | ||
| "tokenizerId": "openai", | ||
| "pricingTiers": [ | ||
| { | ||
| "id": "3d8027df-7491-442e-9770-1363ab2452fc_tier_default", | ||
| "name": "Standard", | ||
| "isDefault": true, | ||
| "priority": 0, | ||
| "conditions": [], | ||
| "prices": { | ||
| "input": 0.75e-6, | ||
| "input_cached_tokens": 0.075e-6, | ||
| "input_cache_read": 0.075e-6, | ||
| "output": 4.5e-6 | ||
| } | ||
| } | ||
| ] | ||
| }, | ||
| { | ||
| "id": "2d83d130-d0f1-4b5d-be1c-7caf70b8e444", | ||
| "modelName": "gpt-5.4-nano", | ||
| "matchPattern": "(?i)^(openai\\/)?(gpt-5.4-nano)$", | ||
| "createdAt": "2026-03-18T00:00:00.000Z", | ||
| "updatedAt": "2026-03-18T00:00:00.000Z", | ||
| "tokenizerConfig": { | ||
| "tokensPerName": 1, | ||
| "tokenizerModel": "gpt-4", | ||
| "tokensPerMessage": 3 | ||
| }, | ||
| "tokenizerId": "openai", | ||
| "pricingTiers": [ | ||
| { | ||
| "id": "2d83d130-d0f1-4b5d-be1c-7caf70b8e444_tier_default", | ||
| "name": "Standard", | ||
| "isDefault": true, | ||
| "priority": 0, | ||
| "conditions": [], | ||
| "prices": { | ||
| "input": 0.2e-6, | ||
| "input_cached_tokens": 0.02e-6, | ||
| "input_cache_read": 0.02e-6, | ||
| "output": 1.25e-6 | ||
| } | ||
| } | ||
| ] | ||
| }, | ||
| { | ||
| "id": "33c7321b-c56e-45e9-9640-33ba3f5cb4fa", | ||
| "modelName": "gpt-5.4-nano-2026-03-17", | ||
| "matchPattern": "(?i)^(openai\\/)?(gpt-5.4-nano-2026-03-17)$", | ||
| "createdAt": "2026-03-18T00:00:00.000Z", | ||
| "updatedAt": "2026-03-18T00:00:00.000Z", | ||
| "tokenizerConfig": { | ||
| "tokensPerName": 1, | ||
| "tokenizerModel": "gpt-4", | ||
| "tokensPerMessage": 3 | ||
| }, | ||
| "tokenizerId": "openai", | ||
| "pricingTiers": [ | ||
| { | ||
| "id": "33c7321b-c56e-45e9-9640-33ba3f5cb4fa_tier_default", | ||
| "name": "Standard", | ||
| "isDefault": true, | ||
| "priority": 0, | ||
| "conditions": [], | ||
| "prices": { | ||
| "input": 0.2e-6, | ||
| "input_cached_tokens": 0.02e-6, | ||
| "input_cache_read": 0.02e-6, | ||
| "output": 1.25e-6 | ||
| } | ||
| } | ||
| ] | ||
| }, |
There was a problem hiding this comment.
Add the missing reasoning price aliases for the new GPT-5.4 mini/nano entries.
Every other gpt-5* record in this table maps output_reasoning_tokens and output_reasoning to the same rate as output, but these four omit both keys. If usage for GPT-5.4 mini/nano is emitted under reasoning-specific details, those tokens will currently price at zero. Re-run pnpm run sync-prices after fixing the source data so the generated TypeScript stays aligned.
Representative fix — apply the same addition to all four new entries
"prices": {
"input": 0.75e-6,
"input_cached_tokens": 0.075e-6,
"input_cache_read": 0.075e-6,
- "output": 4.5e-6
+ "output": 4.5e-6,
+ "output_reasoning_tokens": 4.5e-6,
+ "output_reasoning": 4.5e-6
}Run the following to confirm which gpt-5* entries are missing the reasoning aliases:
#!/bin/bash
set -euo pipefail
python - <<'PY'
import json
from pathlib import Path
path = Path("internal-packages/llm-pricing/src/default-model-prices.json")
models = json.loads(path.read_text())
for model in models:
name = model["modelName"]
if not name.startswith("gpt-5"):
continue
prices = model["pricingTiers"][0]["prices"]
missing = [key for key in ("output_reasoning_tokens", "output_reasoning") if key not in prices]
if missing:
print(f"{name}: missing {', '.join(missing)}")
PY🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@internal-packages/llm-pricing/src/default-model-prices.json` around lines
3787 - 3898, The four new GPT-5.4 entries (modelName "gpt-5.4-mini",
"gpt-5.4-mini-2026-03-17", "gpt-5.4-nano", "gpt-5.4-nano-2026-03-17") are
missing the reasoning price aliases; update each pricingTiers[0].prices object
to add "output_reasoning_tokens" and "output_reasoning" with the same numeric
value as the existing "output" key (so reasoning tokens/pricing match output),
then re-run "pnpm run sync-prices" to regenerate TypeScript artifacts.
1 participant
prompts.define()Prompt management
Define prompts in your code with
prompts.define(), then manage versions and overrides from the dashboard without redeploying:The prompts list page shows each prompt with its current version, model, override status, and a usage sparkline over the last 24 hours.
From the prompt detail page you can:
prompt.resolve()is called.AI span inspectors
Every AI SDK operation now gets a custom inspector in the run trace view:
ai.generateText/ai.streamText— Shows model, token usage, cost, the full message thread (system prompt, user message, assistant response), and linked prompt detailsai.generateObject/ai.streamObject— Same as above plus the JSON schema and structured outputai.toolCall— Shows tool name, call ID, and input argumentsai.embed— Shows model and the text being embeddedFor generation spans linked to a prompt, a "Prompt" tab shows the prompt metadata, the input variables passed to
resolve(), and the template content from the prompt version.All AI span inspectors include a compact timestamp and duration header.
Other improvements
@window-splitter/stateto fix snapshot restoration)<div>inside<p>DOM nesting warnings in span titles and chat messagesScreenshots