Sahar/perf legacy changes by sfatimar · Pull Request #963 · intel/onnxruntime

sfatimar · 2026-03-10T07:01:59Z

Description

https://jira.devtools.intel.com/browse/EISW-195635
This change allows profiling information to be dumped to a Directory defined by environment variable
ORT_OPENVINO_PERF_COUNT when Perf Count Property is enabled.

Motivation and Context

https://jira.devtools.intel.com/browse/EISW-195635
Compiler team has requested for profiling information

Copilot

Pull request overview

Adds support in the OpenVINO EP to dump OpenVINO profiling (perf count) information to a directory specified by the ORT_OPENVINO_PERF_COUNT environment variable when profiling is enabled on the compiled model.

Changes:

Make IBackend::Infer non-const and update BasicBackend accordingly.
Add ORT_OPENVINO_PERF_COUNT env var plumbing and write perf counts to a CSV file under the specified directory.
Simplify printPerformanceCounts APIs and change output format to CSV.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
onnxruntime/core/providers/openvino/ibackend.h	Adjust `Infer` interface constness to support new inference-time behavior.
onnxruntime/core/providers/openvino/backends/basic_backend.h	Update `Infer` override signature to match interface change.
onnxruntime/core/providers/openvino/backends/basic_backend.cc	Implement perf-count CSV dumping to an env-var directory after inference when profiling is enabled.
onnxruntime/core/providers/openvino/backend_utils.h	Add env-var helper and update perf-count printing function signatures.
onnxruntime/core/providers/openvino/backend_utils.cc	Implement env-var helper and update perf-count printing implementation/output format.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-24T07:43:51Z

onnxruntime/core/providers/openvino/backends/basic_backend.cc

+      std::filesystem::create_directory(dir, ec);
+      if (ec) {
+        LOGS_DEFAULT(INFO) << log_tag << ec.message();
+        return;
+      } else {
+        std::string filestring = subgraph_context_.subgraph_name.substr(std::string("OpenVINOExecutionProvider_OpenVINO-EP").length()) + "_perf_count.csv";
+        std::filesystem::path filename(session_context_.GetModelPath().stem().string() + filestring);
+        filename = dir / filename;
+        std::ofstream out(filename);
+        if (out.is_open()) {
+          printPerformanceCounts(infer_request, out);
+          out.close();
+        }
+      }


Using std::filesystem::create_directory(dir, ec) will fail for nested paths (e.g., /tmp/a/b) and doesn’t verify that an existing path is actually a directory. Consider using create_directories and/or checking exists(dir) + is_directory(dir) so ORT_OPENVINO_PERF_COUNT can point to a deeper dump directory reliably.

Suggested change

std::filesystem::create_directory(dir, ec);

if (ec) {

LOGS_DEFAULT(INFO) << log_tag << ec.message();

return;

} else {

std::string filestring = subgraph_context_.subgraph_name.substr(std::string("OpenVINOExecutionProvider_OpenVINO-EP").length()) + "_perf_count.csv";

std::filesystem::path filename(session_context_.GetModelPath().stem().string() + filestring);

filename = dir / filename;

std::ofstream out(filename);

if (out.is_open()) {

printPerformanceCounts(infer_request, out);

out.close();

}

}

// Ensure the performance count path exists and is a directory.

bool exists = std::filesystem::exists(dir, ec);

if (ec) {

LOGS_DEFAULT(INFO) << log_tag << "Failed to check existence of perf count path '" << dir.string() << "': " << ec.message();

return;

}

if (exists) {

bool is_dir = std::filesystem::is_directory(dir, ec);

if (ec || !is_dir) {

LOGS_DEFAULT(INFO) << log_tag << "Perf count path '" << dir.string() << "' exists but is not a directory.";

return;

}

} else {

std::filesystem::create_directories(dir, ec);

if (ec) {

LOGS_DEFAULT(INFO) << log_tag << "Failed to create perf count directory '" << dir.string() << "': " << ec.message();

return;

}

}

std::string filestring = subgraph_context_.subgraph_name.substr(std::string("OpenVINOExecutionProvider_OpenVINO-EP").length()) + "_perf_count.csv";

std::filesystem::path filename(session_context_.GetModelPath().stem().string() + filestring);

filename = dir / filename;

std::ofstream out(filename);

if (out.is_open()) {

printPerformanceCounts(infer_request, out);

out.close();

}

Copilot · 2026-03-24T07:43:51Z

onnxruntime/core/providers/openvino/backends/basic_backend.cc

+        std::filesystem::path filename(session_context_.GetModelPath().stem().string() + filestring);
+        filename = dir / filename;
+        std::ofstream out(filename);
+        if (out.is_open()) {
+          printPerformanceCounts(infer_request, out);
+          out.close();
+        }


This writes to a deterministic filename per model/subgraph on every inference. Since ORT sessions can run concurrently, multiple threads can race and truncate/overwrite the same CSV, producing corrupt output. Please make the output path unique per inference (e.g., include PID/TID/timestamp/request counter) or guard file writing with a mutex; also log when ofstream fails to open so failures aren’t silent.

Copilot · 2026-03-24T07:43:51Z

onnxruntime/core/providers/openvino/backend_utils.h

@@ -72,6 +72,8 @@ namespace backend_utils {

 bool IsDebugEnabled();



IsPerfCountEnabled() returns a string path (not a boolean) so the name is misleading. Consider renaming to something like GetPerfCountDumpDir()/GetPerfCountDumpPath() to better communicate the contract to callers.

Suggested change

// Returns the path where performance counts are dumped when enabled.

std::string GetPerfCountDumpPath();

[[deprecated("IsPerfCountEnabled() is misleadingly named; use GetPerfCountDumpPath() instead.")]]

Copilot · 2026-03-24T07:43:51Z

onnxruntime/core/providers/openvino/backend_utils.cc

 }

+std::string IsPerfCountEnabled() {
+  std::string env_name = onnxruntime::GetEnvironmentVar("ORT_OPENVINO_PERF_COUNT");


IsPerfCountEnabled() re-reads the environment variable on every call, unlike the other helpers in this file that cache via static std::string. Since this is called from the inference path, please cache the value (and reuse the same static pattern) to avoid repeated getenv/alloc work per inference.

Suggested change

std::string env_name = onnxruntime::GetEnvironmentVar("ORT_OPENVINO_PERF_COUNT");

static std::string env_name = onnxruntime::GetEnvironmentVar("ORT_OPENVINO_PERF_COUNT");

Copilot · 2026-03-24T07:43:52Z

onnxruntime/core/providers/openvino/backends/basic_backend.cc

+        LOGS_DEFAULT(INFO) << log_tag << ec.message();
+        return;
+      } else {
+        std::string filestring = subgraph_context_.subgraph_name.substr(std::string("OpenVINOExecutionProvider_OpenVINO-EP").length()) + "_perf_count.csv";


subgraph_context_.subgraph_name.substr(std::string("OpenVINOExecutionProvider_OpenVINO-EP").length()) can throw std::out_of_range if the subgraph name doesn't start with that exact prefix (e.g., OpenVINO subgraph names elsewhere are OpenVINO-EP-subgraph_N). Please guard this with a prefix check (or avoid hardcoding the prefix) so dumping perf counts can’t crash inference.

Suggested change

std::string filestring = subgraph_context_.subgraph_name.substr(std::string("OpenVINOExecutionProvider_OpenVINO-EP").length()) + "_perf_count.csv";

const std::string prefix = "OpenVINOExecutionProvider_OpenVINO-EP";

std::string subgraph_suffix;

if (subgraph_context_.subgraph_name.size() >= prefix.size() &&

subgraph_context_.subgraph_name.compare(0, prefix.size(), prefix) == 0) {

subgraph_suffix = subgraph_context_.subgraph_name.substr(prefix.size());

} else {

// Fallback: use the full subgraph name if it does not start with the expected prefix

subgraph_suffix = subgraph_context_.subgraph_name;

}

std::string filestring = subgraph_suffix + "_perf_count.csv";

ankitm3k · 2026-03-24T08:13:38Z

onnxruntime/core/providers/openvino/backend_utils.cc

 }

+std::string IsPerfCountEnabled() {
+  std::string env_name = onnxruntime::GetEnvironmentVar("ORT_OPENVINO_PERF_COUNT");


Use GetEnvironmentVariableA as GetEnvironmentVar might create env creation conflicts on Windows.

Refer PR - https://github.com/intel-innersource/frameworks.ai.onnxruntime.openvino-plugin-ep/pull/182

onnxruntime/core/providers/openvino/backends/basic_backend.cc

sfatimar added 3 commits March 9, 2026 11:59

COmmitted perf changes on local branch

b5fe94b

Enabling Logging

8bd7b8b

Merge branch 'ovep-develop' into sahar/perf_legacy_changes

acc44c6

sfatimar requested review from ankitm3k and preetha-intel March 24, 2026 07:38

Merge branch 'ovep-develop' into sahar/perf_legacy_changes

7831c3d

ankitm3k requested a review from Copilot March 24, 2026 07:39

Copilot started reviewing on behalf of ankitm3k March 24, 2026 07:39 View session

Copilot AI reviewed Mar 24, 2026

View reviewed changes

ankitm3k reviewed Mar 24, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sahar/perf legacy changes#963

Sahar/perf legacy changes#963
sfatimar wants to merge 4 commits intoovep-developfrom
sahar/perf_legacy_changes

sfatimar commented Mar 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 24, 2026

Uh oh!

Copilot AI Mar 24, 2026

Uh oh!

Copilot AI Mar 24, 2026

Uh oh!

Copilot AI Mar 24, 2026

Uh oh!

Copilot AI Mar 24, 2026

Uh oh!

ankitm3k Mar 24, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		@@ -72,6 +72,8 @@ namespace backend_utils {

		bool IsDebugEnabled();

+// Returns the path where performance counts are dumped when enabled.
+std::string GetPerfCountDumpPath();
+[[deprecated("IsPerfCountEnabled() is misleadingly named; use GetPerfCountDumpPath() instead.")]]

	std::string env_name = onnxruntime::GetEnvironmentVar("ORT_OPENVINO_PERF_COUNT");
	static std::string env_name = onnxruntime::GetEnvironmentVar("ORT_OPENVINO_PERF_COUNT");

-        std::string filestring = subgraph_context_.subgraph_name.substr(std::string("OpenVINOExecutionProvider_OpenVINO-EP").length()) + "_perf_count.csv";
+        const std::string prefix = "OpenVINOExecutionProvider_OpenVINO-EP";
+        std::string subgraph_suffix;
+        if (subgraph_context_.subgraph_name.size() >= prefix.size() &&
+            subgraph_context_.subgraph_name.compare(0, prefix.size(), prefix) == 0) {
+          subgraph_suffix = subgraph_context_.subgraph_name.substr(prefix.size());
+        } else {
+          // Fallback: use the full subgraph name if it does not start with the expected prefix
+          subgraph_suffix = subgraph_context_.subgraph_name;
+        }
+        std::string filestring = subgraph_suffix + "_perf_count.csv";

Conversation

sfatimar commented Mar 10, 2026

Description

Motivation and Context

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

ankitm3k Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants