Skip to content

[Data migration script] Misc improvements#2585

Open
ouvreboite wants to merge 8 commits intolangfuse:mainfrom
ouvreboite:main
Open

[Data migration script] Misc improvements#2585
ouvreboite wants to merge 8 commits intolangfuse:mainfrom
ouvreboite:main

Conversation

@ouvreboite
Copy link
Copy Markdown

@ouvreboite ouvreboite commented Mar 3, 2026

Incremental improvements of the migration notebook:

  • handle unknown observation types (tools/agent/...) as span
  • ensure that traces related to a migrated dataset run are migrated (even if the trace migration has been skipped)
  • make the trace migration idempotent: if the trace is already present in the destination (i.e. same trace id), it is skipped
  • use the from/to date params during the dataset run migration: this allows to only migrate recent dataset runs
  • ensure the relative ordering of the dataset runs is kept: the createdAt are still changed, but at least their relative order is consistent
  • keep the observation and score ids instead of recreating new random ones
  • url encode the run names manually when calling GET /api/public/datasets/{datasetName}/runs/{runName}, mitigating [Python] Path parameters are not url encoded fern-api/fern#13065 while it is fixed
  • rely on fern's built-in rate limit handling (reducing non-business code)
  • use tqdm to display meaningful progress bars for each step
image - simplify and normalize the configuration: the keys and date ranges are configured in the top level cell

@vercel
Copy link
Copy Markdown

vercel Bot commented Mar 3, 2026

Someone is attempting to deploy a commit to the langfuse Team on Vercel.

A member of the Team first needs to authorize it.

@review-notebook-app
Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@dosubot dosubot Bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Mar 3, 2026
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 3, 2026

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ ouvreboite
❌ jb.muscat


jb.muscat seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@dosubot dosubot Bot added the enhancement New feature or request label Mar 3, 2026
@ouvreboite
Copy link
Copy Markdown
Author

Force pushed to use my github account as author

@dosubot dosubot Bot added size:XXL This PR changes 1000+ lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Mar 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:XXL This PR changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants