feat: add instrumentation for openai gem by xuan-cao-swi · Pull Request #1797 · open-telemetry/opentelemetry-ruby-contrib

xuan-cao-swi · 2025-11-24T17:36:39Z

Description

This PR introduces OpenTelemetry instrumentation for the openai Ruby gem from official OpenAI Ruby SDK, enabling automatic tracing for OpenAI API calls including chat completions, streaming responses, and embeddings. The main idea is to monkey-patch the request function which every openai api call use this function with different path.

The captured content is the content of message from user and assistants. It will only log the content if it's enabled.

request function takes all different openai ruby gem api call, but for now it only support chat (with stream) and embeddings

Chat Completion

client = OpenAI::Client.new(api_key: ENV['OPENAI_API_KEY'])

response = client.chat.completions.create(
  model: 'gpt-5-nano',
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'Explain what Ruby is in one sentence.' }
  ],
  max_completion_tokens: 1000
)

# Result and Captured Content (if enabled):
I, [2025-11-22T00:04:48.965241 #99703]  INFO -- : {"event":"gen_ai.system.message","attributes":{"gen_ai.provider.name":"openai"},"body":{"content":"You are a helpful assistant."}}
I, [2025-11-22T00:04:48.965263 #99703]  INFO -- : {"event":"gen_ai.user.message","attributes":{"gen_ai.provider.name":"openai"},"body":{"content":"Explain what Ruby is in one sentence."}}
I, [2025-11-22T00:04:52.589295 #99703]  INFO -- : {"event":"gen_ai.choice","attributes":{"gen_ai.provider.name":"openai"},"body":{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"Ruby is a dynamic, open-source programming language designed for simplicity and productivity, with an elegant, human-friendly syntax."}}}

"Ruby is a dynamic, open-source programming language designed for simplicity and productivity, with an elegant, human-friendly syntax."

# Captured Span Attributes
-> gen_ai.operation.name: Str(chat)
-> gen_ai.provider.name: Str(openai)
-> gen_ai.request.model: Str(gpt-5-nano)
-> server.address: Str(api.openai.com)
-> server.port: Int(443)
-> http.request.method: Str(POST)
-> url.path: Str(chat/completions)
-> gen_ai.output.type: Str(text)
-> gen_ai.request.max_tokens: Int(1000)
-> gen_ai.response.model: Str(gpt-5-nano-2025-08-07)
-> gen_ai.response.id: Str(chatcmpl-CeVRmR6ubrhgLJOvFPvoDzm7uq0zm)
-> openai.response.service_tier: Str(default)
-> gen_ai.usage.input_tokens: Int(24)
-> gen_ai.usage.output_tokens: Int(232)
-> gen_ai.usage.total_tokens: Int(256)
-> gen_ai.response.finish_reasons: Slice(["stop"])

Streaming Chat Completion

stream = client.chat.completions.stream(
  model: 'gpt-5-nano',
  messages: [{ role: 'user', content: 'Count from 1 to 5 and say done.' }],
  max_completion_tokens: 1000
)

# Result and Captured Content (if enabled):
I, [2025-11-22T00:07:30.620543 #1730]  INFO -- : {"event":"gen_ai.system.message","attributes":{"gen_ai.provider.name":"openai"},"body":{"content":"You are a helpful assistant."}}
I, [2025-11-22T00:07:30.620575 #1730]  INFO -- : {"event":"gen_ai.user.message","attributes":{"gen_ai.provider.name":"openai"},"body":{"content":"Count from 1 to 5 and say done."}}
1, 2, 3, 4, 5. Done.
[Finished: stop]
I, [2025-11-22T00:07:33.966164 #1730]  INFO -- : {"event":"gen_ai.choice","attributes":{"gen_ai.provider.name":"openai"},"body":{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"1, 2, 3, 4, 5. Done."}}}

# Captured Span Attributes
-> gen_ai.operation.name: Str(chat)
-> gen_ai.provider.name: Str(openai)
-> gen_ai.request.model: Str(gpt-5-nano)
-> server.address: Str(api.openai.com)
-> server.port: Int(443)
-> http.request.method: Str(POST)
-> url.path: Str(chat/completions)
-> gen_ai.output.type: Str(text)
-> gen_ai.request.max_tokens: Int(1000)
-> gen_ai.response.model: Str(gpt-5-nano-2025-08-07)
-> gen_ai.response.id: Str(chatcmpl-CeVRzNeMmotPgE4f76N9DKAfS6z9g)
-> gen_ai.usage.input_tokens: Int(27)
-> gen_ai.usage.output_tokens: Int(217)
-> gen_ai.response.finish_reasons: Slice(["stop"])
-> openai.response.service_tier: Str(default)

Text Embeddings

response = client.embeddings.create(
  model: 'text-embedding-3-small',
  input: 'Ruby is a dynamic, open source programming language.'
)

# Result and Captured Content (if enabled):
I, [2025-11-22T00:08:40.827714 #2458]  INFO -- : {"event":"gen_ai.user.message","attributes":{"gen_ai.provider.name":"openai"},"body":{"content":"Ruby is a dynamic, open source programming language."}}
First 5 values: [-0.0451, 0.0145, -0.0182, -0.0009, 0.0348]
Usage: {:prompt_tokens=>10, :total_tokens=>10}
Embedding dimensions: 1536

# Captured Span Attributes
-> gen_ai.operation.name: Str(embeddings)
-> gen_ai.provider.name: Str(openai)
-> gen_ai.request.model: Str(text-embedding-3-small)
-> server.address: Str(api.openai.com)
-> server.port: Int(443)
-> http.request.method: Str(POST)
-> url.path: Str(embeddings)
-> gen_ai.output.type: Str(json)
-> gen_ai.response.model: Str(text-embedding-3-small)
-> gen_ai.usage.input_tokens: Int(10)
-> gen_ai.usage.total_tokens: Int(10)
-> gen_ai.embeddings.dimension.count: Int(1536)

arielvalentin · 2025-11-25T23:32:03Z

@xuan-cao-swi did you generate this using instrumentation_generator ? There are a few changes missing like the toys release metadata, adding it to CI, patching the all gem. etc...

github-actions · 2025-12-26T02:12:17Z

👋 This pull request has been marked as stale because it has been open with no activity. You can: comment on the issue or remove the stale label to hold stale off for a while, add the keep label to hold stale off permanently, or do nothing. If you do nothing this pull request will be closed eventually by the stale bot

xuan-cao-swi · 2026-01-07T04:20:39Z

@arielvalentin yes, I did use instrumentation_generator. I ran the command again, I saw the changes

modified:   .github/component_owners.yml
modified:   .toys/.data/releases.yml
modified:   instrumentation/all/Gemfile
modified:   instrumentation/all/lib/opentelemetry/instrumentation/all.rb

Are we starting to include these before making the release of new gem?

arielvalentin · 2026-01-07T05:20:00Z

Yes. All of those changes should be included by default unless you want to explicitly omit this instrumentation from all and defer releasing this gem.

…etry-ruby-contrib into openai-ruby-instr

xuan-cao-swi · 2026-01-07T15:14:25Z

Done.

github-actions · 2026-02-07T02:13:31Z

👋 This pull request has been marked as stale because it has been open with no activity. You can: comment on the issue or remove the stale label to hold stale off for a while, add the keep label to hold stale off permanently, or do nothing. If you do nothing this pull request will be closed eventually by the stale bot

arielvalentin · 2026-04-09T04:32:11Z

@xuan-cao-swi can you sync this with main?

arielvalentin · 2026-04-18T14:55:55Z

@xuan-cao-swi can I ask you to address broken tests?

thompson-tomo · 2026-04-22T11:21:41Z

+  gem 'opentelemetry-instrumentation-base', path: '../base'
+  gem 'appraisal', '~> 2.5'
+  gem 'bundler', '~> 2.4'
+  gem 'minitest', '~> 5.0'
+  gem 'openai', '~> 0.36.0'
+  gem 'opentelemetry-sdk', '~> 1.0'
+  gem 'opentelemetry-test-helpers', '~> 0.3'
+  gem 'rake', '~> 13.0'
+  gem 'rubocop', '~> 1.81.1'
+  gem 'rubocop-minitest', '~> 0.39.0'
+  gem 'rubocop-rspec', '~> 3.9.0'
+  gem 'rubocop-rake', '~> 0.7.1'
+  gem 'rubocop-performance', '~> 1.26.0'
+  gem 'simplecov', '~> 0.17.1'
+  gem 'webmock', '~> 3.24'
+  gem 'yard', '~> 0.9'


Suggested change

gem 'opentelemetry-instrumentation-base', path: '../base'

gem 'appraisal', '~> 2.5'

gem 'bundler', '~> 2.4'

gem 'minitest', '~> 5.0'

gem 'openai', '~> 0.36.0'

gem 'opentelemetry-sdk', '~> 1.0'

gem 'opentelemetry-test-helpers', '~> 0.3'

gem 'rake', '~> 13.0'

gem 'rubocop', '~> 1.81.1'

gem 'rubocop-minitest', '~> 0.39.0'

gem 'rubocop-rspec', '~> 3.9.0'

gem 'rubocop-rake', '~> 0.7.1'

gem 'rubocop-performance', '~> 1.26.0'

gem 'simplecov', '~> 0.17.1'

gem 'webmock', '~> 3.24'

gem 'yard', '~> 0.9'

gem 'opentelemetry-instrumentation-base', path: '../base'

gem 'appraisal', '~> 2.5.0'

gem 'bundler', '~> 2.4.0'

gem 'minitest', '~> 5.0'

gem 'openai', '~> 0.36.0'

gem 'opentelemetry-sdk', '~> 1.0.0'

gem 'opentelemetry-test-helpers', '~> 0.3.0'

gem 'rake', '~> 13.0.0'

gem 'rubocop', '~> 1.81.1'

gem 'rubocop-minitest', '~> 0.39.0'

gem 'rubocop-rspec', '~> 3.9.0'

gem 'rubocop-rake', '~> 0.7.1'

gem 'rubocop-performance', '~> 1.26.0'

gem 'simplecov', '~> 0.17.1'

gem 'webmock', '~> 3.24.0'

gem 'yard', '~> 0.9.0'

thompson-tomo · 2026-04-22T11:32:06Z

+    module OpenAI
+      module Patches
+        # OpenAIClient Patch
+        # rubocop:disable  Metrics/ModuleLength


Can we move this to the rubocop config file

thompson-tomo · 2026-04-22T11:34:12Z

+              'gen_ai.request.model' => model,
+              'server.address' => uri&.host || 'api.openai.com',
+              'server.port' => uri&.port || 443,
+              'http.request.method' => req[:method].to_s.upcase,


Suggested change

'http.request.method' => req[:method].to_s.upcase,

thompson-tomo · 2026-04-22T11:35:03Z

+              'server.address' => uri&.host || 'api.openai.com',
+              'server.port' => uri&.port || 443,
+              'http.request.method' => req[:method].to_s.upcase,
+              'url.path' => req[:path],


Suggested change

'url.path' => req[:path],

xuan-cao-swi and others added 5 commits November 21, 2025 14:15

feat: openai instrumentation

ce48949

remove old notes/comments

e745ea1

update readme

8ae5558

fix typo

2825d9a

Merge branch 'main' into openai-ruby-instr

480dbd5

xuan-cao-swi marked this pull request as ready for review November 25, 2025 19:01

xuan-cao-swi requested review from a team, ahayworth, arielvalentin, dazuma, ericmustin, fbogsany, kaylareopelle, mwear, plantfansam, robbkidd, robertlaurin and simi as code owners November 25, 2025 19:01

github-actions Bot added the stale Marks an issue/PR stale label Dec 26, 2025

merge

4a81eac

xuan-cao-swi added 2 commits January 7, 2026 10:13

add to release ci and all

91ae8c8

Merge branch 'openai-ruby-instr' of github.com:xuan-cao-swi/opentelem…

aa3c98b

…etry-ruby-contrib into openai-ruby-instr

github-actions Bot assigned xuan-cao-swi Jan 7, 2026

github-actions Bot removed the stale Marks an issue/PR stale label Jan 8, 2026

xuan-cao-swi mentioned this pull request Jan 12, 2026

Add support for GenAI instrumentation #1786

Closed

github-actions Bot added the stale Marks an issue/PR stale label Feb 7, 2026

github-actions Bot closed this Mar 9, 2026

xuan-cao-swi reopened this Mar 17, 2026

xuan-cao-swi requested a review from hannahramadan as a code owner March 17, 2026 13:48

github-actions Bot added ci and removed stale Marks an issue/PR stale labels Mar 17, 2026

Merge branch 'main' into openai-ruby-instr

3be50bb

xuan-cao-swi and others added 2 commits April 20, 2026 11:02

Merge branch 'main' into openai-ruby-instr

f37f57b

update test; update for 4.0

971e337

thompson-tomo reviewed Apr 22, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add instrumentation for openai gem#1797

feat: add instrumentation for openai gem#1797
xuan-cao-swi wants to merge 11 commits intoopen-telemetry:mainfrom
xuan-cao-swi:openai-ruby-instr

xuan-cao-swi commented Nov 24, 2025

Uh oh!

arielvalentin commented Nov 25, 2025

Uh oh!

github-actions Bot commented Dec 26, 2025

Uh oh!

xuan-cao-swi commented Jan 7, 2026

Uh oh!

arielvalentin commented Jan 7, 2026

Uh oh!

xuan-cao-swi commented Jan 7, 2026

Uh oh!

github-actions Bot commented Feb 7, 2026

Uh oh!

arielvalentin commented Apr 9, 2026

Uh oh!

arielvalentin commented Apr 18, 2026

Uh oh!

thompson-tomo Apr 22, 2026

Uh oh!

thompson-tomo Apr 22, 2026

Uh oh!

thompson-tomo Apr 22, 2026

Uh oh!

thompson-tomo Apr 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

xuan-cao-swi commented Nov 24, 2025

Description

Chat Completion

Streaming Chat Completion

Text Embeddings

Uh oh!

arielvalentin commented Nov 25, 2025

Uh oh!

github-actions Bot commented Dec 26, 2025

Uh oh!

xuan-cao-swi commented Jan 7, 2026

Uh oh!

arielvalentin commented Jan 7, 2026

Uh oh!

xuan-cao-swi commented Jan 7, 2026

Uh oh!

github-actions Bot commented Feb 7, 2026

Uh oh!

arielvalentin commented Apr 9, 2026

Uh oh!

arielvalentin commented Apr 18, 2026

Uh oh!

thompson-tomo Apr 22, 2026

Choose a reason for hiding this comment

Uh oh!

thompson-tomo Apr 22, 2026

Choose a reason for hiding this comment

Uh oh!

thompson-tomo Apr 22, 2026

Choose a reason for hiding this comment

Uh oh!

thompson-tomo Apr 22, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants