fix: copilot by using bucketId and adjust agent models by joanagmaia · Pull Request #1691 · linuxfoundation/insights

joanagmaia · 2026-02-19T14:21:09Z

This pull request introduces significant improvements to the Data Copilot's agent orchestration and data auditing, focusing on model selection, Tinybird integration, and auditor prompt accuracy. The changes enable more precise use of language models for different agents, ensure correct data partitioning with Tinybird's bucketId, and enhance the auditor's ability to provide user-relevant summaries by including actual data samples.

Model selection and agent orchestration:

Refactored the DataCopilot class to use two separate Bedrock models: Sonnet for routing and pipe agents, and Opus for text-to-SQL and auditor agents, allowing for more optimal model usage per agent type.

Tinybird integration and data partitioning:

Added logic to fetch and cache the Tinybird bucketId per project, injecting it into all relevant Tinybird pipe calls and tool executions to ensure correct data partitioning and prevent cross-project data leakage.

Auditor prompt and data summary improvements:

Enhanced the auditor prompt to include the top rows of actual data (not just statistics), requiring the summary to reference these real values and handle unknown or placeholder entries appropriately.
Adjusted the auditor output schema to allow feedback_to_router and summary to be nullable, reflecting cases where these may not be present.

Other improvements:

Filtered out Tinybird tools with empty descriptions to prevent Bedrock validation errors.
Minor bug fix: ensured previousFeedback is set to undefined if not present, improving retry logic robustness.

These changes collectively improve the reliability, accuracy, and user-friendliness of the Data Copilot's responses and its integration with the Tinybird data backend.

Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

Copilot

Pull request overview

This PR updates the AWS Bedrock model identifier from Claude Sonnet 4 (us.anthropic.claude-sonnet-4-20250514-v1:0) to Claude Opus 4-6 (us.anthropic.claude-opus-4-6-v1) across all usage locations in the chat/data copilot system. This represents both a model upgrade (from Sonnet to Opus) and a change in the model identifier format.

Changes:

Updated Bedrock model identifier from Sonnet 4 to Opus 4-6 across all code and test files
Updated documentation to reflect the new model configuration
Maintained consistency across production code, tests, and documentation

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
frontend/lib/chat/data-copilot.ts	Updated BEDROCK_MODEL_ID constant in main DataCopilot class
frontend/lib/chat/chart/generator.ts	Updated model identifier for chart generation functionality
frontend/lib/chat/tests/router.test.ts	Updated model identifier in router agent tests
frontend/lib/chat/tests/auditor.test.ts	Updated model identifier in auditor agent tests
frontend/lib/chat/Readme.md	Updated documentation to reflect new model configuration

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

frontend/lib/chat/data-copilot.ts

Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

joanagmaia · 2026-02-25T19:27:59Z

@epipav can you check it out and let me know what you think? — Summary of the changes in the PR description.

The copilot was quite broken in production, so I made a set of changes to address the main issues.

The primary problem was that bucketId logic was never implemented. As a result, most requests were returning 0 because the system couldn’t properly resolve projects (this is the current production behavior). This PR introduces the missing bucketId handling to ensure queries are correctly scoped.

I also evaluated using Opus 4.6 across all agents, but it proved too slow—particularly for the Router and Pipe agents, where it would often stall or take too long to respond. Given the current architecture, it doesn’t seem viable to use it there without more significant changes, so I’m now using one model for some agents, and opus model for others.

Additionally, I improved the data summary returned to the user by including the top 3 rows. This increases token usage slightly, but the improvement in response clarity and UX is noticeable. While this won’t cover every edge case, it works well for common queries like “most active X” and similar straightforward requests.

Let me know if anything here doesn’t look right—especially since you have more context on some parts of the system.

Some tests:

epipav

Looks good overall - have u tried using Sonnet 4.6?

epipav

Thanks for fixing the bucket issue! Added nitpicks and few readablity comments

epipav · 2026-02-26T10:57:50Z

frontend/lib/chat/data-copilot.ts

+      // For now, we use the Opus model for text-to-SQL and auditor.
+      // The model is currently too slow for both the pipe and router agents.


readability NIT: This comment block should come before let model: string | undefined = this.BEDROCK_SONNET_MODEL_ID

epipav · 2026-02-26T11:01:59Z

frontend/lib/chat/data-copilot.ts


-  /** Amazon Bedrock language model instance */
-  private model: LanguageModelV1;
+  /** Amazon Bedrock language model instance for routing and auditing (Sonnet) */


Auditor uses Opus, not Sonnet

epipav · 2026-02-26T11:03:10Z

frontend/lib/chat/data-copilot.ts

+  /** Amazon Bedrock language model instance for routing and auditing (Sonnet) */
+  private sonnetModel: LanguageModelV1;
+
+  /** Amazon Bedrock language model instance for text-to-SQL, pipe, and chart agents (Opus) */


docs look wrong: pipe agent uses Sonnet, not Opus

epipav · 2026-02-26T11:03:57Z

frontend/lib/chat/utils/data-summary.ts

 * Generate statistical summary of dataset
- * Token-efficient: ~400-500 tokens for typical dataset
- * No raw data samples sent to LLM - only statistics
+ * Token-efficient: ~1500-200 tokens for typical dataset


There's a typo here, should be 1500-2000

epipav · 2026-02-26T11:07:51Z

frontend/lib/chat/instructions.ts

    try {
-      const result = await executeTinybirdPipe(pipeInstruction.name, pipeInstruction.inputs);
+      const inputs =
+        bucketId !== null ? { bucketId, ...pipeInstruction.inputs } : pipeInstruction.inputs;


The optional bucketId param can also be undefined here when the function call doesn't have bucketId. Checking with loose equality (bucketId != null) also catches undefined here

epipav · 2026-02-26T11:08:34Z

frontend/lib/chat/data-copilot.ts

+        { headers: { Authorization: `Bearer ${tinybirdToken}` }, timeout: 10_000 },
+      );
+      this.bucketId = response.data?.[0]?.bucketId ?? null;
+      console.warn(`🪣 [DataCopilot] bucketId for "${project}": ${this.bucketId}`);


NIT: let's use console.log here, since it's a non-warning log

epipav · 2026-02-26T11:11:45Z

frontend/lib/chat/utils/data-summary.ts

+    // Ideally the entire response would be available to the auditor. But that would be too costly.
+    // TODO: Explore a better way to have a proper summary of the data that answers the user's question directly.


NIT: This TODO comment doesn't add much here - Data summary already tries to tackle this somewhat, and if it needs better summarization, I think we can create a task for it

epipav · 2026-02-26T11:21:04Z

frontend/lib/chat/data-copilot.ts

+    const allTools = await this.mcpClient.tools({});
+
+    // Filter out tools with empty descriptions — Bedrock rejects them with a validation error
+    this.tbTools = Object.fromEntries(


Anything important is missing descriptions? Should we report these to Tinybird support?

joanagmaia added 2 commits February 18, 2026 17:28

chore: bump bedrock model for data copilot

18db504

Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

chore: add region prefix to model id

db49be8

Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

Copilot AI review requested due to automatic review settings February 19, 2026 14:21

Copilot started reviewing on behalf of joanagmaia February 19, 2026 14:21 View session

Copilot AI reviewed Feb 19, 2026

View reviewed changes

frontend/lib/chat/data-copilot.ts Outdated Show resolved Hide resolved

joanagmaia added 7 commits February 25, 2026 12:27

chore: test different models

d5dac30

Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

Merge remote-tracking branch 'origin/main' into chore/bump-bedrock-model

5f5c7a4

chore: revert timeout

25595b2

Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

fix: models

01c2483

Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

chore: further improvements

b83314d

Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

fix: readme

76da4d8

Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

fix: imports

22e0e71

Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

joanagmaia requested a review from epipav February 25, 2026 19:19

joanagmaia changed the title ~~chore: bump bedrock model~~ fix: copilot by using bucketId and adjust agent models Feb 25, 2026

epipav reviewed Feb 26, 2026

View reviewed changes

epipav approved these changes Feb 26, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: copilot by using bucketId and adjust agent models#1691

fix: copilot by using bucketId and adjust agent models#1691
joanagmaia wants to merge 9 commits intomainfrom
chore/bump-bedrock-model

joanagmaia commented Feb 19, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

joanagmaia commented Feb 25, 2026

Uh oh!

epipav left a comment •

edited

Loading

Uh oh!

epipav left a comment

Uh oh!

epipav Feb 26, 2026

Uh oh!

epipav Feb 26, 2026

Uh oh!

epipav Feb 26, 2026

Uh oh!

epipav Feb 26, 2026

Uh oh!

epipav Feb 26, 2026

Uh oh!

epipav Feb 26, 2026

Uh oh!

epipav Feb 26, 2026

Uh oh!

epipav Feb 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		// For now, we use the Opus model for text-to-SQL and auditor.
		// The model is currently too slow for both the pipe and router agents.

		// Ideally the entire response would be available to the auditor. But that would be too costly.
		// TODO: Explore a better way to have a proper summary of the data that answers the user's question directly.

Conversation

joanagmaia commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

joanagmaia commented Feb 25, 2026

Let me know if anything here doesn’t look right—especially since you have more context on some parts of the system.

Uh oh!

epipav left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

epipav left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

joanagmaia commented Feb 19, 2026 •

edited

Loading

epipav left a comment •

edited

Loading