feat(sdk): implement exporter metrics

[anuraaga]

Description

I am helping to implement SDK internal metrics

https://opentelemetry.io/docs/specs/semconv/otel/sdk-metrics/

This implements the exporter metrics.

Similar PR in JS: open-telemetry/opentelemetry-js#6480

/cc @xrmx to help with review. As there are many exporters, this is a pretty big one

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• Unit tests

Does This PR Require a Contrib Repo Change?

• Yes. - Link to PR:
• No.

Checklist:

• Followed the style guidelines of this project
• Changelogs have been updated
• Unit tests have been added
• Documentation has been updated

[herin049]

Can we make this more consistent with other places in the codebase? For example in opentelemetry-sdk/src/opentelemetry/sdk/_configuration/__init__.py we use

Suggested change

	signal: Literal["span", "log", "metric_data_point"],
	signal: Literal["traces", "metrics", "logs"],

[herin049]

Could we instead just have three methods on_start, on_end and on_error (similar to the logging implementation) and use an instance variable or ContextVar for tracking the duration?

[anuraaga]

Not sure if that would simplify things, especially since exceptions are suppressed by the exporter implementations (otherwise a context manager would work great). ContextVar is good for black-box code but seems unnecessary when we are instrumenting the SDK itself. Could return a class, but it seems like it would be very similar to the Callable. Can do that if it seems better though.

[lzchen]

Does using a context manager as a mutable object work to record error info? We can use finally to do metric stuff after return is called in export.

with self._metrics.export_operation(self._count_data(data)) as result:
...
    result.error = error
    result.error_attrs = {RPC_RESPONSE_STATUS_CODE: error.code().name}

and then in ExporterMetrics:

...
  @contextmanager
  def export_operation(self, num_items: int):
       result = ExportResult()
      try:
          yield result
      finally:
           ...
          duration_attrs = result.error_attrs if result.error_attrs else _standard_attrs
      self._duration.record(metric, duration_attrs)

start_export -> finish_export would probably work in practice because the operations in finish are fine to be left dangling if we don't call them in every case (like if exporter shutdown during export) but design wise it seems dangerous to me.

[anuraaga]

Thanks for the idea! I'll give that a try

[anuraaga]

I have applied the cleanups except for the method change since I couldn't see how to do it and make things significantly cleaner.

I will merge main after #5015

[anuraaga]

@xrmx I realized one reason I had it string was to keep component_type an optional parameter for API evolution purposes, that in practice we always set. But I guess this might be ok - I could also assert it is set, but didn't seem worth it

[anuraaga]

Now fails with this which is also somewhat unrelated to this PR. #5030 (review)

self = <opentelemetry.exporter.otlp.proto.http.metric_exporter.OTLPMetricExporter object at 0x1090b3860>, metrics_data = 'foo', timeout_millis = 10000, kwargs = {}
num_items = 0

    def export(
        self,
        metrics_data: MetricsData,
        timeout_millis: Optional[float] = 10000,
        **kwargs,
    ) -> MetricExportResult:
        if self._shutdown:
            _logger.warning("Exporter already shutdown, ignoring batch")
            return MetricExportResult.FAILURE

        num_items = 0
>       for resource_metrics in metrics_data.resource_metrics:
E       AttributeError: 'str' object has no attribute 'resource_metrics'

exporter/opentelemetry-exporter-otlp-proto-http/src/opentelemetry/exporter/otlp/proto/http/metric_exporter/__init__.py:349: AttributeError

Caused by https://github.com/open-telemetry/opentelemetry-python/blob/main/exporter/opentelemetry-exporter-otlp-proto-http/tests/metrics/test_otlp_metrics_exporter.py#L929