Skip to content

fix(crewai): share singleton handler for OpenAI-v2 context inheritance#5

Open
shuningc wants to merge 1 commit intomainfrom
copy-pr-205
Open

fix(crewai): share singleton handler for OpenAI-v2 context inheritance#5
shuningc wants to merge 1 commit intomainfrom
copy-pr-205

Conversation

@shuningc
Copy link
Owner

@shuningc shuningc commented Feb 11, 2026

Switch CrewAI instrumentation to use the shared telemetry handler singleton and add regression coverage that verifies OpenAI-v2 chat spans and metrics inherit CrewAI agent context.

Summary by CodeRabbit

  • Refactor

    • Updated telemetry handler initialization mechanism.
  • Tests

    • Added test coverage for agent identity inheritance when CrewAI and OpenAI integrations share telemetry infrastructure.
    • Updated existing tests to align with refactored handler initialization.

Switch CrewAI instrumentation to use the shared telemetry handler singleton and add regression coverage that verifies OpenAI-v2 chat spans and metrics inherit CrewAI agent context.

Co-authored-by: Cursor <cursoragent@cursor.com>
@qodo-code-review
Copy link

Review Summary by Qodo

Use singleton handler for OpenAI-v2 context inheritance in CrewAI

🐞 Bug fix 🧪 Tests

Grey Divider

Walkthroughs

Description
• Switch CrewAI instrumentation to use shared singleton TelemetryHandler
• Add regression test validating OpenAI-v2 chat spans inherit agent context
• Update test mocks to use get_telemetry_handler instead of direct constructor
• Ensure proper handler cleanup between tests via conftest fixture
Diagram
flowchart LR
  A["CrewAI Instrumentation"] -->|"uses singleton"| B["get_telemetry_handler"]
  B -->|"provides"| C["TelemetryHandler"]
  C -->|"shared by"| D["OpenAI-v2 Chat Spans"]
  D -->|"inherit"| E["Agent Context"]
Loading

Grey Divider

File Changes

1. instrumentation-genai/opentelemetry-instrumentation-crewai/src/opentelemetry/instrumentation/crewai/instrumentation.py ✨ Enhancement +5/-2

Switch to singleton handler factory function

• Import get_telemetry_handler function alongside TelemetryHandler class
• Replace direct TelemetryHandler() constructor call with get_telemetry_handler() to use
 singleton pattern
• Maintains same tracer_provider and meter_provider arguments

instrumentation-genai/opentelemetry-instrumentation-crewai/src/opentelemetry/instrumentation/crewai/instrumentation.py


2. instrumentation-genai/opentelemetry-instrumentation-crewai/tests/conftest.py 🧪 Tests +5/-0

Add singleton handler cleanup in test fixture

• Import get_telemetry_handler in reset_global_handler fixture
• Add cleanup logic to remove _default_handler attribute from singleton before and after each test
• Ensures test isolation by resetting both module-level handler and singleton state

instrumentation-genai/opentelemetry-instrumentation-crewai/tests/conftest.py


3. instrumentation-genai/opentelemetry-instrumentation-crewai/tests/test_openai_v2_inheritance.py 🧪 Tests +154/-0

Add OpenAI-v2 context inheritance regression test

• New comprehensive test validating OpenAI-v2 chat spans inherit CrewAI agent context
• Simulates agent execution with LLMInvocation lifecycle (start_llm/stop_llm)
• Verifies agent name and ID propagate to both spans and metrics
• Validates both duration and token usage metrics contain inherited agent attributes

instrumentation-genai/opentelemetry-instrumentation-crewai/tests/test_openai_v2_inheritance.py


View more (1)
4. instrumentation-genai/opentelemetry-instrumentation-crewai/tests/test_singleton.py 🧪 Tests +11/-11

Update singleton handler mocks in tests

• Update three test methods to mock get_telemetry_handler instead of TelemetryHandler class
• Rename mock variables from MockHandler to mock_get_handler for clarity
• Adjust assertions to verify singleton factory function calls instead of constructor

instrumentation-genai/opentelemetry-instrumentation-crewai/tests/test_singleton.py


Grey Divider

Qodo Logo

@coderabbitai
Copy link

coderabbitai bot commented Feb 11, 2026

📝 Walkthrough

Walkthrough

The changes introduce a factory function pattern for TelemetryHandler creation, replacing direct instantiation with get_telemetry_handler(). Tests are updated to mock this new factory pattern and verify proper handler initialization. A new test validates that CrewAI and OpenAI-v2 share singleton handler state while correctly inheriting agent identity.

Changes

Cohort / File(s) Summary
Main Instrumentation
instrumentation-genai/opentelemetry-instrumentation-crewai/src/opentelemetry/instrumentation/crewai/instrumentation.py
Replaced direct TelemetryHandler construction with factory function get_telemetry_handler(). Updated imports to expose the new factory function.
Test Fixtures
instrumentation-genai/opentelemetry-instrumentation-crewai/tests/conftest.py
Extended global handler reset logic to manage _default_handler attribute on the factory function alongside existing handler reset behavior.
Test Updates
instrumentation-genai/opentelemetry-instrumentation-crewai/tests/test_singleton.py
Updated mocks to target get_telemetry_handler factory function instead of direct TelemetryHandler class. Assertions now verify factory function calls with provider arguments.
New Test
instrumentation-genai/opentelemetry-instrumentation-crewai/tests/test_openai_v2_inheritance.py
Added comprehensive test verifying that CrewAI and OpenAI-v2 instrumentation share a singleton TelemetryHandler, correctly propagating agent identity (name and ID) through chat spans and token-usage metrics.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Poem

🐰 A factory for handlers, so neat and so keen,
No more direct construction on this scene,
Singletons shared 'tween CrewAI and friends,
Agent identities dance from start to end,
One hop, one skip, telemetry's blessed! ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 71.43% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: switching to a shared singleton handler for OpenAI-v2 context inheritance, which aligns with the PR's core objective of enabling context sharing between CrewAI and OpenAI-v2 instrumentation.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch copy-pr-205

No actionable comments were generated in the recent review. 🎉

🧹 Recent nitpick comments
instrumentation-genai/opentelemetry-instrumentation-crewai/tests/test_openai_v2_inheritance.py (4)

24-26: Redundant singleton reset — autouse fixture already handles this.

The reset_global_handler autouse fixture in conftest.py (Lines 54–55) already deletes _default_handler before each test. This manual reset is harmless but unnecessary.

♻️ Suggested removal
-    # Ensure fresh singleton with test providers.
-    if hasattr(get_telemetry_handler, "_default_handler"):
-        delattr(get_telemetry_handler, "_default_handler")
-

152-154: Manual cleanup conflicts with / duplicates autouse fixtures.

crewai_module._handler = None is already handled by the reset_global_handler fixture teardown. The os.environ.pop("OTEL_INSTRUMENTATION_GENAI_EMITTERS", None) on Line 154 is managed by the environment fixture. Although no bug results (fixture teardown still restores correctly), this can mislead readers into thinking the fixtures don't cover cleanup. Consider removing these lines.

♻️ Suggested removal
-    # Keep environment clean for neighboring tests that may inspect globals.
-    crewai_module._handler = None
-    os.environ.pop("OTEL_INSTRUMENTATION_GENAI_EMITTERS", None)

84-88: Swallowing force_flush errors may hide flaky test failures.

If force_flush() raises, no metrics will be flushed, and the assertions below (Lines 111–150) will fail with an opaque AssertionError rather than the root cause. Consider letting the exception propagate or at least logging it.

♻️ Let the exception propagate for clearer diagnostics
-    try:
-        meter_provider.force_flush()
-    except Exception:
-        pass
-    metric_reader.collect()
+    meter_provider.force_flush()
+    metric_reader.collect()

90-92: Direct indexing into resource_metrics[0].scope_metrics[0].metrics assumes a single scope.

This will raise IndexError if the metrics pipeline produces an unexpected structure (e.g., multiple scopes). In a focused unit test this is usually fine, but if this test is ever run with additional instrumentations active, it could break in a confusing way. A defensive lookup (iterate and match scope name) would be more robust, but this is acceptable for now.

Tip

Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@qodo-code-review
Copy link

Code Review by Qodo

🐞 Bugs (2) 📘 Rule violations (1) 📎 Requirement gaps (0)

Grey Divider


Action required

1. Singleton ignores providers 🐞 Bug ✓ Correctness
Description
CrewAI now uses the shared get_telemetry_handler singleton; once any instrumentation initializes the
singleton, later calls with different tracer/meter providers are silently ignored, making telemetry
configuration order-dependent across instrumentations (e.g., OpenAI-v2 vs CrewAI). This can lead to
spans/metrics being emitted through unexpected providers/exporters with no warning.
Code

instrumentation-genai/opentelemetry-instrumentation-crewai/src/opentelemetry/instrumentation/crewai/instrumentation.py[R114-116]

+        _handler = get_telemetry_handler(
            tracer_provider=tracer_provider, meter_provider=meter_provider
        )
Evidence
CrewAI passes tracer/meter providers into get_telemetry_handler, but get_telemetry_handler only uses
those parameters when creating the singleton the first time. OpenAI-v2 also calls
get_telemetry_handler with providers, so whichever instrumentor runs first effectively locks in the
providers for both instrumentations thereafter.

instrumentation-genai/opentelemetry-instrumentation-crewai/src/opentelemetry/instrumentation/crewai/instrumentation.py[97-116]
util/opentelemetry-util-genai/src/opentelemetry/util/genai/handler.py[1115-1133]
instrumentation-genai/opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/init.py[68-79]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`get_telemetry_handler()` caches a singleton `TelemetryHandler` and ignores later `tracer_provider`/`meter_provider`/`logger_provider` arguments once `_default_handler` is set. After this PR, CrewAI also uses that singleton, so the first instrumentor to run effectively “wins” provider configuration for all other instrumentations that share the singleton.

### Issue Context
- CrewAI now does `_handler = get_telemetry_handler(tracer_provider=..., meter_provider=...)`.
- OpenAI-v2 also does `handler = get_telemetry_handler(tracer_provider=..., meter_provider=..., logger_provider=...)`.
- `get_telemetry_handler()` only constructs the handler on first call and returns the cached instance thereafter.

### Fix Focus Areas
- util/opentelemetry-util-genai/src/opentelemetry/util/genai/handler.py[1115-1133]
- instrumentation-genai/opentelemetry-instrumentation-crewai/src/opentelemetry/instrumentation/crewai/instrumentation.py[97-116]
- instrumentation-genai/opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/__init__.py[68-79]

### Suggested approach
- Add a supported API path for deterministic initialization (e.g., `initialize_telemetry_handler(...)` / `set_default_telemetry_handler(handler)`), and have instrumentors use it.
- Additionally (or if you don’t want a new API), add mismatch detection in `get_telemetry_handler()`:
 - If a cached handler exists and any provider argument is non-None, emit a WARNING that the singleton is already configured and the passed providers are being ignored.
 - Document the “first-call wins” semantics clearly if that remains the intended behavior.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

2. force_flush() exceptions swallowed 📘 Rule violation ⛯ Reliability
Description
The test silently ignores any failure from meter_provider.force_flush(), which can mask real
telemetry pipeline regressions and make failures harder to diagnose. Catch a narrower exception (or
fail the test) with actionable context instead of except Exception: pass.
Code

instrumentation-genai/opentelemetry-instrumentation-crewai/tests/test_openai_v2_inheritance.py[R84-87]

+    try:
+        meter_provider.force_flush()
+    except Exception:
+        pass
Evidence
PR Compliance ID 3 requires robust error handling and explicitly calls out silent failures/swallowed
exceptions as failure criteria; the new test code catches a broad exception and does nothing, hiding
errors.

Rule 3: Generic: Robust Error Handling and Edge Case Management
instrumentation-genai/opentelemetry-instrumentation-crewai/tests/test_openai_v2_inheritance.py[84-87]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The test currently does `except Exception: pass` around `meter_provider.force_flush()`, which silently hides failures and makes telemetry regressions harder to detect.

## Issue Context
Compliance requires robust error handling and explicitly disallows swallowed exceptions without context/logging.

## Fix Focus Areas
- instrumentation-genai/opentelemetry-instrumentation-crewai/tests/test_openai_v2_inheritance.py[84-87]ేంద

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Test assumes first metric scope 🐞 Bug ⛯ Reliability
Description
The new inheritance test only inspects resource_metrics[0].scope_metrics[0], so it can throw
IndexError or miss the expected gen-ai metrics if they are recorded under a different scope/resource
within the same MeterProvider. This can cause false test failures when additional scopes exist.
Code

instrumentation-genai/opentelemetry-instrumentation-crewai/tests/test_openai_v2_inheritance.py[R90-93]

+    resource_metrics = metric_reader.get_metrics_data().resource_metrics
+    assert resource_metrics
+    metrics = resource_metrics[0].scope_metrics[0].metrics
+
Evidence
The test asserts resource_metrics is non-empty but then unconditionally indexes into the first
scope (scope_metrics[0]) and only searches metrics within that single scope. If the gen-ai metrics
are present under a different scope (or if the first scope list is empty), the test will fail even
though metrics exist.

instrumentation-genai/opentelemetry-instrumentation-crewai/tests/test_openai_v2_inheritance.py[84-100]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The test hard-codes `resource_metrics[0].scope_metrics[0].metrics`. This assumes the first resource and first scope always contain the gen-ai metrics, which may not hold if multiple metric scopes/resources are present.

### Issue Context
Even though the test searches by metric name afterward, it only searches within the single selected scope.

### Fix Focus Areas
- instrumentation-genai/opentelemetry-instrumentation-crewai/tests/test_openai_v2_inheritance.py[90-109]

### Suggested approach
- Replace the single-scope extraction with a flattening search, e.g.:
 - iterate over `for rm in resource_metrics: for sm in rm.scope_metrics: for metric in sm.metrics:`
 - collect metrics into a dict by name, or just find the first match by name across all scopes
- Optionally add assertions that `scope_metrics` exists if you still index into it.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

ⓘ The new review experience is currently in Beta. Learn more

Grey Divider

Qodo Logo

Comment on lines +114 to 116
_handler = get_telemetry_handler(
tracer_provider=tracer_provider, meter_provider=meter_provider
)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Singleton ignores providers 🐞 Bug ✓ Correctness

CrewAI now uses the shared get_telemetry_handler singleton; once any instrumentation initializes the
singleton, later calls with different tracer/meter providers are silently ignored, making telemetry
configuration order-dependent across instrumentations (e.g., OpenAI-v2 vs CrewAI). This can lead to
spans/metrics being emitted through unexpected providers/exporters with no warning.
Agent Prompt
### Issue description
`get_telemetry_handler()` caches a singleton `TelemetryHandler` and ignores later `tracer_provider`/`meter_provider`/`logger_provider` arguments once `_default_handler` is set. After this PR, CrewAI also uses that singleton, so the first instrumentor to run effectively “wins” provider configuration for all other instrumentations that share the singleton.

### Issue Context
- CrewAI now does `_handler = get_telemetry_handler(tracer_provider=..., meter_provider=...)`.
- OpenAI-v2 also does `handler = get_telemetry_handler(tracer_provider=..., meter_provider=..., logger_provider=...)`.
- `get_telemetry_handler()` only constructs the handler on first call and returns the cached instance thereafter.

### Fix Focus Areas
- util/opentelemetry-util-genai/src/opentelemetry/util/genai/handler.py[1115-1133]
- instrumentation-genai/opentelemetry-instrumentation-crewai/src/opentelemetry/instrumentation/crewai/instrumentation.py[97-116]
- instrumentation-genai/opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/__init__.py[68-79]

### Suggested approach
- Add a supported API path for deterministic initialization (e.g., `initialize_telemetry_handler(...)` / `set_default_telemetry_handler(handler)`), and have instrumentors use it.
- Additionally (or if you don’t want a new API), add mismatch detection in `get_telemetry_handler()`:
  - If a cached handler exists and any provider argument is non-None, emit a WARNING that the singleton is already configured and the passed providers are being ignored.
  - Document the “first-call wins” semantics clearly if that remains the intended behavior.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants