Add OpenTelemetry instrumentation for LiteLLM#88
Merged
Cirilla-zmh merged 8 commits intoalibaba:mainfrom Mar 10, 2026
Merged
Conversation
|
|
github-actions
bot
requested review from
123liuziming,
Cirilla-zmh and
ralf0131
whoIam0987
force-pushed
the
mingzhi/litellm
branch
from
c26059b to
d8affed
Compare
whoIam0987
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
This pull request adds OpenTelemetry instrumentation for the LiteLLM library, which provides a unified interface to 100+ LLM providers. The instrumentation captures telemetry data for LLM operations including completions, embeddings, streaming, tool calls, and retry mechanisms.
Changes:
- Added new instrumentation package for LiteLLM with support for sync/async completion and embedding APIs
- Implemented streaming response wrappers with proper span lifecycle management
- Added comprehensive test suite covering various LiteLLM features including tool calls, retries, and error handling
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 19 comments.
Show a summary per file
| File | Description |
|---|---|
__init__.py |
Main instrumentor class that wraps LiteLLM functions and manages telemetry handlers |
_wrapper.py |
Completion wrappers for sync/async calls with streaming support |
_embedding_wrapper.py |
Embedding operation wrappers for sync/async calls |
_stream_wrapper.py |
Stream response wrappers handling chunk accumulation and finalization |
_utils.py |
Utility functions for message conversion, provider parsing, and invocation creation |
version.py, package.py |
Package metadata and dependency declarations |
pyproject.toml |
Package configuration with build system and dependencies |
README.rst |
Documentation for installation, configuration, and usage |
test_*.py |
Comprehensive test suite covering completions, embeddings, streaming, tools, retries, and errors |
test-requirements.txt |
Test dependency specifications |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
...uite-instrumentation-litellm/src/opentelemetry/instrumentation/litellm/_embedding_wrapper.py
Outdated
Show resolved
Hide resolved
...uite-instrumentation-litellm/src/opentelemetry/instrumentation/litellm/_embedding_wrapper.py
Outdated
Show resolved
Hide resolved
...ite/loongsuite-instrumentation-litellm/src/opentelemetry/instrumentation/litellm/_wrapper.py
Outdated
Show resolved
Hide resolved
...ite/loongsuite-instrumentation-litellm/src/opentelemetry/instrumentation/litellm/_wrapper.py
Show resolved
Hide resolved
...ite/loongsuite-instrumentation-litellm/src/opentelemetry/instrumentation/litellm/__init__.py
Outdated
Show resolved
Hide resolved
...ite/loongsuite-instrumentation-litellm/src/opentelemetry/instrumentation/litellm/_wrapper.py
Outdated
Show resolved
Hide resolved
...ite/loongsuite-instrumentation-litellm/src/opentelemetry/instrumentation/litellm/_wrapper.py
Show resolved
Hide resolved
...ite/loongsuite-instrumentation-litellm/src/opentelemetry/instrumentation/litellm/_wrapper.py
Outdated
Show resolved
Hide resolved
...ite/loongsuite-instrumentation-litellm/src/opentelemetry/instrumentation/litellm/_wrapper.py
Outdated
Show resolved
Hide resolved
...ite/loongsuite-instrumentation-litellm/src/opentelemetry/instrumentation/litellm/_wrapper.py
Outdated
Show resolved
Hide resolved
whoIam0987
force-pushed
the
mingzhi/litellm
branch
from
d8affed to
002d956
Compare
Cirilla-zmh
requested changes
Feb 13, 2026
Collaborator
Cirilla-zmh
left a comment
Cirilla-zmh
left a comment
There was a problem hiding this comment.
Thanks for the contribution! Please address the remaining comments so we can move this PR forward.
instrumentation-loongsuite/loongsuite-instrumentation-litellm/test-requirements.txt
Outdated
Show resolved
Hide resolved
instrumentation-loongsuite/loongsuite-instrumentation-litellm/pyproject.toml
Outdated
Show resolved
Hide resolved
instrumentation-loongsuite/loongsuite-instrumentation-litellm/test-requirements.txt
Outdated
Show resolved
Hide resolved
instrumentation-loongsuite/loongsuite-instrumentation-litellm/tests/test_error_handling.py
Show resolved
Hide resolved
whoIam0987
force-pushed
the
mingzhi/litellm
branch
from
487a9f4 to
cdce6e7
Compare
Cirilla-zmh
requested changes
Mar 4, 2026
Collaborator
Cirilla-zmh
left a comment
Cirilla-zmh
left a comment
There was a problem hiding this comment.
Some comments should be resolved.
instrumentation-loongsuite/loongsuite-instrumentation-litellm/pyproject.toml
Outdated
Show resolved
Hide resolved
instrumentation-loongsuite/loongsuite-instrumentation-litellm/LICENSE
Outdated
Show resolved
Hide resolved
instrumentation-loongsuite/loongsuite-instrumentation-litellm/README.rst
Outdated
Show resolved
Hide resolved
...suite/loongsuite-instrumentation-litellm/src/opentelemetry/instrumentation/litellm/_utils.py
Outdated
Show resolved
Hide resolved
...uite/loongsuite-instrumentation-litellm/src/opentelemetry/instrumentation/litellm/package.py
Outdated
Show resolved
Hide resolved
Add litellm tests to GitHub workflow.
2 This LISENSE file has been removed; 3 The CHANGELOG.md file has been added; 4 This SUPPRESS_LLM_SDK_KEY and releted code have been removed; 5 Metric is not supported.
whoIam0987
force-pushed
the
mingzhi/litellm
branch
from
3f4c47e to
f207c4f
Compare
Cirilla-zmh
added
enhancement
Change-Id: I5f4ff4d48fbafd665e9f8b6e6049e870ff3c102b Co-developed-by: Cursor <noreply@cursor.com>
123liuziming
approved these changes
Mar 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Instrument Litellm with genai util.
Fixes #60 (issue)
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Does This PR Require a Core Repo Change?
Checklist:
See contributing.md for styleguide, changelog guidelines, and more.