Skip to content

[Major Branch][REBASE&FF] Patina Test framework changes #1377

Merged
Javagedes merged 2 commits intoOpenDevicePartnership:majorfrom
Javagedes:personal/joeyvagedes/patina-test-major-branch
Mar 10, 2026
Merged

[Major Branch][REBASE&FF] Patina Test framework changes #1377
Javagedes merged 2 commits intoOpenDevicePartnership:majorfrom
Javagedes:personal/joeyvagedes/patina-test-major-branch

Conversation

@Javagedes
Copy link
Copy Markdown
Contributor

@Javagedes Javagedes commented Mar 6, 2026

Description

This commit chain makes two breaking changes to the Patina Test framework:

  1. Updates the .with_filter function to accept a Filter enum instead of a string, to better enable filtering multiple ways (right now only allow list and deny list)
  2. Moves patina_test framework out of the patina crate and into it's own patina component crate. As a part of this change, the build-lib makefile command has been updated to pass -all-features. This change has already been pushed on main in patina-devops.
Commit 1 (Click to expand commit message) Add exclude filter support to TestRunner Introduce a `Filter` enum with `Include` and `Exclude` variants rather than a generic string that is only an inclusion filter.

Exclude filters take priority over include filters. If no include
filters are specified, all non-excluded tests run. The match arms in
should_run are exhaustive to ensure new variants cause a compile break.

Commit 2 (Click to expand commit message) move patina_test framework to patina_test crate The patina_test framework was originally developed inside the SDK crate to keep `patina_macro` as only a dependency for the `patina` crate. As we have evolved, other patina related crates have taken a dependency on `patina_macro`, which means the patina_test framework can be it's own crate.

This has one large benefit: It simplfies feature logic related to
actually enable or disable the tests. There is now a single feature flag
that is off by default (component), and this is how test writers will
consume the crate. When a platform wants to register a test runner with
the Core, it will enable the component feature flag, which does two
things: (1) Make the TestRunner component public and (2) enable a
feature flag in patina_macro to actually register all found patina
tests globally.

The second, smaller benefit, is that we are moving some of the non-sdk
logic out of our SDK crate, and this framework makes sense as a
standalone crate.

  • Impacts functionality?
  • Impacts security?
  • Breaking change?
  • Includes tests?
  • Includes documentation?

How This Was Tested

CI, Platform verification

Integration Instructions

.with_filter update

// Previous
TestRunner::default()
  .with_filter("whitelisted::filter")

// New
TestRunner::default()
  .with_filter(Filter::include("include::this"))
  .with_filter(Filter::exclude("exclude::this"))

New crate update

// Previous
use patina::test::{TestRunner, patina_test, u_assert, u_assert_eq, u_assert_ne, Result};

// New
use patina_test::{component::TestRunner, error::Result, u_assert, u_assert_eq, u_asssert_ne, patina_test};
# Old
[dependencies]
patina = { features = "enable_patina_tests"] }

# new
[dependencies]
patina_test = { features = ["component"] }

@patina-automation
Copy link
Copy Markdown
Contributor

patina-automation Bot commented Mar 6, 2026

QEMU Validation Failed

QEMU validation did not complete successfully or did not shutdown as expected.

Workflow run: https://github.com/OpenDevicePartnership/patina/actions/runs/22914037405

⚠️ Build Failure Detected

The Setup and Build patina-dxe-core-qemu step failed in the following job(s):

  • Run Patina QEMU Validation / Validate QEMU - Q35 (Linux)
  • Run Patina QEMU Validation / Validate QEMU Q35 (Windows)
  • Run Patina QEMU Validation / Validate QEMU - SBSA (Linux)

Build logs are available in the workflow run artifacts:

Download the log artifacts from the workflow run to review the build output and
determine whether the error is expected.

Is this expected? If the PR introduces breaking changes to patina that
require a corresponding update to patina-dxe-core-qemu,
the build failure is expected.

Contact a repo admin or PR reviewer to override the Patina QEMU PR Validation status check.

Job Results

Job Result
Gather Incoming PR Metadata
Run Patina QEMU Validation / Preflight Checks
Run Patina QEMU Validation / Post In-Progress Notification
Run Patina QEMU Validation / Get Constants / Get Repository Constants
Run Patina QEMU Validation / Validate QEMU - Q35 (Linux)
Run Patina QEMU Validation / Validate QEMU Q35 (Windows)
Run Patina QEMU Validation / Validate QEMU - SBSA (Linux)
Run Patina QEMU Validation / Emit PR Metadata

Error Details

qemu-validation-logs-Linux-Q35/q35-linux.log (2 error/warning sections)
error: failed to select a version for `patina`.
    ... required by package `qemu_dxe_core v2.2.13 (/__w/patina/deps/patina-dxe-core-qemu)`

[cargo-make] ERROR - Error while running duckscript: Source: Unknown Line: 109 - Error while executing command, exit code: 101
qemu-validation-logs-Windows-Q35/q35-windows.log (2 error/warning sections)
error: failed to select a version for `patina`.
    ... required by package `qemu_dxe_core v2.2.13 (D:\a\patina\deps\patina-dxe-core-qemu)`

[cargo-make] ERROR - Error while running duckscript: Source: Unknown Line: 109 - Error while executing command, exit code: 101
qemu-validation-logs-Linux-SBSA/sbsa-linux.log (2 error/warning sections)
error: failed to select a version for `patina`.
    ... required by package `qemu_dxe_core v2.2.13 (/__w/patina/deps/patina-dxe-core-qemu)`

[cargo-make] ERROR - Error while running duckscript: Source: Unknown Line: 109 - Error while executing command, exit code: 101

This comment was automatically generated by the Patina QEMU PR Validation Post workflow.

@github-actions github-actions Bot added impact:breaking-change Requires integration attention impact:non-functional Does not have a functional impact impact:testing Affects testing type:documentation Improvements or additions to documentation labels Mar 6, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 6, 2026

Codecov Report

❌ Patch coverage is 94.35897% with 11 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
components/patina_test/src/service.rs 94.03% 8 Missing ⚠️
components/patina_test/src/__private_api.rs 93.75% 1 Missing ⚠️
components/patina_test/src/component.rs 97.50% 1 Missing ⚠️
patina_dxe_core/src/memory_manager.rs 0.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@Javagedes Javagedes force-pushed the personal/joeyvagedes/patina-test-major-branch branch 3 times, most recently from e2d7226 to 2313a0c Compare March 6, 2026 19:57
@Javagedes Javagedes requested review from cfernald, makubacki and os-d March 6, 2026 20:05
Introduce a `Filter` enum with `Include` and `Exclude` variants rather
than a generic string that is only an inclusion filter.

Exclude filters take priority over include filters. If no include
filters are specified, all non-excluded tests run. The match arms in
`should_run` are exhaustive to ensure new variants cause a compile break.
Comment thread components/patina_test/src/component.rs Outdated
Comment thread components/patina_test/Cargo.toml Outdated
Comment thread components/patina_test/src/component.rs
Comment thread Makefile.toml
Comment thread components/patina_test/src/service.rs Outdated
@Javagedes Javagedes force-pushed the personal/joeyvagedes/patina-test-major-branch branch 3 times, most recently from 2c9c1cb to 65a1970 Compare March 10, 2026 15:58
The patina_test framework was originally developed inside the SDK crate
to keep `patina_macro` as only a dependency for the `patina` crate. As
we have evolved, other patina related crates have taken a dependency on
`patina_macro`, which means the patina_test framework can be it's own
crate.

This has one large benefit: It simplfies feature logic related to
actually enable or disable the tests. There is now a single feature flag
that is off by default (`test-runner`), and this is how test writers will
consume the crate. When a platform wants to register a test runner with
the Core, it will enable the `test-runner` feature flag, which does two
things: (1) Make the `TestRunner` component public and (2) enable a
feature flag in `patina_macro` to actually register all found patina
tests globally.

The second, smaller benefit, is that we are moving some of the non-sdk
logic out of our SDK crate, and this framework makes sense as a
standalone crate.
@Javagedes Javagedes force-pushed the personal/joeyvagedes/patina-test-major-branch branch from 65a1970 to d09e561 Compare March 10, 2026 16:39
@Javagedes Javagedes merged commit 7cd0bcc into OpenDevicePartnership:major Mar 10, 2026
9 checks passed
Javagedes added a commit to Javagedes/patina that referenced this pull request Mar 27, 2026
Introduce a `Filter` enum with `Include` and `Exclude` variants rather
than a generic string that is only an inclusion filter.

Exclude filters take priority over include filters. If no include
filters are specified, all non-excluded tests run. The match arms in
`should_run` are exhaustive to ensure new variants cause a compile break.
Javagedes added a commit to Javagedes/patina that referenced this pull request Mar 27, 2026
…p#1377)

The patina_test framework was originally developed inside the SDK crate
to keep `patina_macro` as only a dependency for the `patina` crate. As
we have evolved, other patina related crates have taken a dependency on
`patina_macro`, which means the patina_test framework can be it's own
crate.

This has one large benefit: It simplfies feature logic related to
actually enable or disable the tests. There is now a single feature flag
that is off by default (`test-runner`), and this is how test writers will
consume the crate. When a platform wants to register a test runner with
the Core, it will enable the `test-runner` feature flag, which does two
things: (1) Make the `TestRunner` component public and (2) enable a
feature flag in `patina_macro` to actually register all found patina
tests globally.

The second, smaller benefit, is that we are moving some of the non-sdk
logic out of our SDK crate, and this framework makes sense as a
standalone crate.
Javagedes added a commit to Javagedes/patina that referenced this pull request Mar 27, 2026
Introduce a `Filter` enum with `Include` and `Exclude` variants rather
than a generic string that is only an inclusion filter.

Exclude filters take priority over include filters. If no include
filters are specified, all non-excluded tests run. The match arms in
`should_run` are exhaustive to ensure new variants cause a compile break.
Javagedes added a commit to Javagedes/patina that referenced this pull request Mar 27, 2026
…p#1377)

The patina_test framework was originally developed inside the SDK crate
to keep `patina_macro` as only a dependency for the `patina` crate. As
we have evolved, other patina related crates have taken a dependency on
`patina_macro`, which means the patina_test framework can be it's own
crate.

This has one large benefit: It simplfies feature logic related to
actually enable or disable the tests. There is now a single feature flag
that is off by default (`test-runner`), and this is how test writers will
consume the crate. When a platform wants to register a test runner with
the Core, it will enable the `test-runner` feature flag, which does two
things: (1) Make the `TestRunner` component public and (2) enable a
feature flag in `patina_macro` to actually register all found patina
tests globally.

The second, smaller benefit, is that we are moving some of the non-sdk
logic out of our SDK crate, and this framework makes sense as a
standalone crate.
Javagedes added a commit that referenced this pull request Mar 27, 2026
Introduce a `Filter` enum with `Include` and `Exclude` variants rather
than a generic string that is only an inclusion filter.

Exclude filters take priority over include filters. If no include
filters are specified, all non-excluded tests run. The match arms in
`should_run` are exhaustive to ensure new variants cause a compile break.
Javagedes added a commit that referenced this pull request Mar 27, 2026
The patina_test framework was originally developed inside the SDK crate
to keep `patina_macro` as only a dependency for the `patina` crate. As
we have evolved, other patina related crates have taken a dependency on
`patina_macro`, which means the patina_test framework can be it's own
crate.

This has one large benefit: It simplfies feature logic related to
actually enable or disable the tests. There is now a single feature flag
that is off by default (`test-runner`), and this is how test writers will
consume the crate. When a platform wants to register a test runner with
the Core, it will enable the `test-runner` feature flag, which does two
things: (1) Make the `TestRunner` component public and (2) enable a
feature flag in `patina_macro` to actually register all found patina
tests globally.

The second, smaller benefit, is that we are moving some of the non-sdk
logic out of our SDK crate, and this framework makes sense as a
standalone crate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

impact:breaking-change Requires integration attention impact:non-functional Does not have a functional impact impact:testing Affects testing type:documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Reorganize the enable_patina_test Feature to be Owned by patina_dxe_core

3 participants