fix(opentelemetry lib): decode missing scope, schema_url, and resource fields

[szibis]

Summary

The OTLP decode path silently drops several protobuf fields during conversion to Vector events, causing data loss and breaking round-trip fidelity (OTLP → Vector → OTLP).

Before this PR

Field	Logs	Traces	Metrics
scope.name	✅	❌	✅ (tag)
scope.version	✅	❌	✅ (tag)
scope.attributes	✅	❌	✅ (tags)
scope.dropped_attributes_count	✅	❌	❌
ScopeX.schema_url	❌	❌	❌
ResourceX.schema_url	❌	❌	❌
resource.dropped_attributes_count	❌	❌	❌

After this PR

Field	Logs	Traces	Metrics
scope.name	✅	✅	✅ (tag)
scope.version	✅	✅	✅ (tag)
scope.attributes	✅	✅	✅ (tags)
scope.dropped_attributes_count	✅	✅	✅ (tag)
ScopeX.schema_url	✅	✅	✅ (tag)
ResourceX.schema_url	✅	✅	✅ (tag)
resource.dropped_attributes_count	✅	✅	✅ (tag)

Field mapping

Logs (Legacy / Vector namespace):

• ScopeLogs.schema_url → scope.schema_url / %opentelemetry.scope.schema_url
• ResourceLogs.schema_url → schema_url / %opentelemetry.resources.schema_url
• Resource.dropped_attributes_count → resource_dropped_attributes_count / %opentelemetry.resources.dropped_attributes_count

Traces (always at event root):

• ScopeSpans.scope.* → scope.name, scope.version, scope.attributes, scope.dropped_attributes_count
• ScopeSpans.schema_url → scope.schema_url
• ResourceSpans.schema_url → schema_url
• Resource.dropped_attributes_count → resource_dropped_attributes_count

Metrics (as tags, following existing resource.* / scope.* pattern):

• scope.dropped_attributes_count, scope.schema_url, resource.schema_url, resource.dropped_attributes_count

Related

• Closes OpenTelemetry source: trace decode drops scope, schema_url fields #24904 (trace decode drops scope and schema_url)
• Relates to [opentelemetry source] Update mapping of OpenTelemetry log data model to LogEvent #15500 (log data model mapping gaps)
• Companion to feat(opentelemetry sink): add automatic native log and trace to OTLP conversion #24621 (native log/trace → OTLP encoding)
• Companion to feat(opentelemetry sink): add native metric to OTLP conversion #24897 (native metrics → OTLP encoding)

Test plan

• 31 new unit tests covering all signal types (12 logs, 11 traces, 8 metrics)
• Tests verify both presence of new fields and absence when empty/zero
• Tests cover Legacy and Vector namespace for logs
• Combined tests verify all new fields work together with existing fields
• Integration test with OTLP collector for round-trip verification

[pront]

Hi @szibis, you have quite a few OTEL PRs open: https://github.com/vectordotdev/vector/pulls?q=sort%3Aupdated-desc+is%3Apr+is%3Aopen+author%3Aszibis+

Can you please list the order in which you want me to review them here? Even better, I would mark all but one as draft so I can keep filtering them without need to exchange comments here.

[szibis]

Hi @szibis, you have quite a few OTEL PRs open: https://github.com/vectordotdev/vector/pulls?q=sort%3Aupdated-desc+is%3Apr+is%3Aopen+author%3Aszibis+

Can you please list the order in which you want me to review them here? Even better, I would mark all but one as draft so I can keep filtering them without need to exchange comments here.

@pront Sorry for that, but I just discovered this gaps and avoiding one big PR addon.

1. fix(opentelemetry lib): decode missing scope, schema_url, and resource fields #24905 - this PR for decode missing scopes - Full OTLP format baseline for all later PR's
2. feat(opentelemetry sink): add automatic native log and trace to OTLP conversion #24621 - For auto-convert sink in Logs and Traces
3. feat(opentelemetry sink): add native metric to OTLP conversion #24897 - Based on Logs and Traces native convert implement Metrics auto convert.

[pront]

I think the new resources.schema_url / resources.dropped_attributes_count field handling introduces a backwards-compatibility issue with Vector namespace logs: they’re written into the same resources object that already contains arbitrary OTLP resource attributes, so valid incoming attributes can now be silently overwritten.

From my local test:

{
  "service.name": "checkout",
  "schema_url": "tenant-defined-value",
  "dropped_attributes_count": "user-payload"
}

With resource metadata:

{
  "schema_url": "https://resource.schema",
  "dropped_attributes_count": 7
}

The emitted event was:

{
  "otel_resources": {
    "service.name": "checkout",
    "schema_url": "https://resource.schema",
    "dropped_attributes_count": 7
  }
}

So the original resource attributes schema_url = "tenant-defined-value" and dropped_attributes_count = "user-payload" were lost.

Repro config:

data_dir: "/tmp/vector-pr-24905-data"

sources:
  otel:
    type: opentelemetry
    use_otlp_decoding: false
    log_namespace: true
    grpc:
      address: "127.0.0.1:43171"
    http:
      address: "127.0.0.1:43181"

transforms:
  expose_meta:
    type: remap
    inputs:
      - otel.logs
    source: |
      .otel_resources = %opentelemetry.resources
      .otel_scope = %opentelemetry.scope
      .otel_timestamp = %opentelemetry.timestamp

sinks:
  out:
    type: console
    inputs:
      - expose_meta
    target: stdout
    encoding:
      codec: json

We should preserve %opentelemetry.resources as the raw user-supplied resource attributes.

Also, we have the same type of issue with Metrics: resource.* / scope.* tag collisions.

Generally, if a field was not literally present as a user attribute, it should not be inserted into the raw attribute map. We should not place synthetic or derived metadata into a namespace that is also used for raw user payload.

[szibis]

We should preserve %opentelemetry.resources as the raw user-supplied resource attributes.

Also, we have the same type of issue with Metrics: resource.* / scope.* tag collisions.

Generally, if a field was not literally present as a user attribute, it should not be inserted into the raw attribute map. We should not place synthetic or derived metadata into a namespace that is also used for raw user payload.

@pront All fixed