MINIFICPP-2644 Add proxy controller service support for GCP processors#2112
MINIFICPP-2644 Add proxy controller service support for GCP processors#2112lordgamez wants to merge 1 commit intoapache:MINIFICPP-2644_aws_azurefrom
Conversation
There was a problem hiding this comment.
Pull request overview
This pull request adds proxy controller service support to GCP processors (PutGCSObject, FetchGCSObject, ListGCSBucket, and DeleteGCSObject). The implementation follows the same pattern used in AWS and Azure processors by adding a "Proxy Configuration Service" property that accepts a ProxyConfigurationServiceInterface controller service.
Changes:
- Added proxy configuration support to the base GCSProcessor class
- Added integration tests for proxy functionality using HTTP and HTTPS proxy types
- Updated processor documentation to include the new proxy configuration property
- Improved container deployment logic to prevent redeployment of already-deployed containers
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| extensions/gcp/processors/GCSProcessor.h | Added ProxyConfigurationService property definition and proxy_ member variable to support proxy configuration |
| extensions/gcp/processors/GCSProcessor.cpp | Implemented proxy configuration parsing in onSchedule() and proxy application in getClient() |
| extensions/gcp/tests/features/google_cloud_storage.feature | Added integration test scenarios for PutGCSObject, FetchGCSObject, ListGCSBucket, and DeleteGCSObject with HTTP and HTTPS proxy support; changed "When the MiNiFi instance starts up" to "When all instances start up" for consistency |
| PROCESSORS.md | Updated documentation for DeleteGCSObject, FetchGCSObject, ListGCSBucket, and PutGCSObject to include the new Proxy Configuration Service property |
| behave_framework/src/minifi_test_framework/containers/container.py | Added is_deployed() method and check to prevent redeployment of already-deployed containers |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| #include "google/cloud/storage/client.h" | ||
| #include "google/cloud/storage/retry_policy.h" | ||
| #include "minifi-cpp/controllers/ProxyConfigurationServiceInterface.h" | ||
| #include "controllers/ProxyConfiguration.h" |
There was a problem hiding this comment.
The include "controllers/ProxyConfiguration.h" is not needed in this header file. The header only uses google::cloud::ProxyConfig (line 79), not minifi::controllers::ProxyConfiguration. This include should be removed to reduce unnecessary dependencies and improve compilation times.
| #include "controllers/ProxyConfiguration.h" |
| .withDescription("Specifies the Proxy Configuration Controller Service to proxy network requests. When used, " | ||
| "this will override any values specified for Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties.") |
There was a problem hiding this comment.
The description for the Proxy Configuration Service property mentions overriding "Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties," but GCP processors do not have these individual proxy properties. Unlike AWS processors which support both individual proxy properties and a proxy configuration service, GCP processors only support the proxy configuration service. Consider updating the description to remove the reference to individual proxy properties, or simply state: "Specifies the Proxy Configuration Controller Service to proxy network requests."
| .withDescription("Specifies the Proxy Configuration Controller Service to proxy network requests. When used, " | |
| "this will override any values specified for Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties.") | |
| .withDescription("Specifies the Proxy Configuration Controller Service to proxy network requests.") |
| | **GCP Credentials Provider Service** | | | The Controller Service used to obtain Google Cloud Platform credentials. Should be the name of a GCPCredentialsControllerService. | | ||
| | **Number of retries** | 6 | | How many retry attempts should be made before routing to the failure relationship. | | ||
| | Endpoint Override URL | | | Overrides the default Google Cloud Storage endpoints<br/>**Supports Expression Language: true** | | ||
| | Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. When used, this will override any values specified for Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties. | |
There was a problem hiding this comment.
The description for the Proxy Configuration Service property mentions overriding "Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties," but GCP processors do not have these individual proxy properties. Unlike AWS processors which support both individual proxy properties and a proxy configuration service, GCP processors only support the proxy configuration service. The description should be updated to remove the reference to non-existent properties, or simply state: "Specifies the Proxy Configuration Controller Service to proxy network requests."
| | Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. When used, this will override any values specified for Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties. | | |
| | Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. | |
| | **GCP Credentials Provider Service** | | | The Controller Service used to obtain Google Cloud Platform credentials. Should be the name of a GCPCredentialsControllerService. | | ||
| | **Number of retries** | 6 | | How many retry attempts should be made before routing to the failure relationship. | | ||
| | Endpoint Override URL | | | Overrides the default Google Cloud Storage endpoints<br/>**Supports Expression Language: true** | | ||
| | Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. When used, this will override any values specified for Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties. | |
There was a problem hiding this comment.
The description for the Proxy Configuration Service property mentions overriding "Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties," but GCP processors do not have these individual proxy properties. Unlike AWS processors which support both individual proxy properties and a proxy configuration service, GCP processors only support the proxy configuration service. The description should be updated to remove the reference to non-existent properties, or simply state: "Specifies the Proxy Configuration Controller Service to proxy network requests."
| | Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. When used, this will override any values specified for Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties. | | |
| | Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. | |
| | **GCP Credentials Provider Service** | | | The Controller Service used to obtain Google Cloud Platform credentials. Should be the name of a GCPCredentialsControllerService. | | ||
| | **Number of retries** | 6 | | How many retry attempts should be made before routing to the failure relationship. | | ||
| | Endpoint Override URL | | | Overrides the default Google Cloud Storage endpoints<br/>**Supports Expression Language: true** | | ||
| | Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. When used, this will override any values specified for Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties. | |
There was a problem hiding this comment.
The description for the Proxy Configuration Service property mentions overriding "Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties," but GCP processors do not have these individual proxy properties. Unlike AWS processors which support both individual proxy properties and a proxy configuration service, GCP processors only support the proxy configuration service. The description should be updated to remove the reference to non-existent properties, or simply state: "Specifies the Proxy Configuration Controller Service to proxy network requests."
| | Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. When used, this will override any values specified for Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties. | | |
| | Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. | |
| | **GCP Credentials Provider Service** | | | The Controller Service used to obtain Google Cloud Platform credentials. Should be the name of a GCPCredentialsControllerService. | | ||
| | **Number of retries** | 6 | | How many retry attempts should be made before routing to the failure relationship. | | ||
| | Endpoint Override URL | | | Overrides the default Google Cloud Storage endpoints<br/>**Supports Expression Language: true** | | ||
| | Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. When used, this will override any values specified for Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties. | |
There was a problem hiding this comment.
The description for the Proxy Configuration Service property mentions overriding "Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties," but GCP processors do not have these individual proxy properties. Unlike AWS processors which support both individual proxy properties and a proxy configuration service, GCP processors only support the proxy configuration service. The description should be updated to remove the reference to non-existent properties, or simply state: "Specifies the Proxy Configuration Controller Service to proxy network requests."
| | Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. When used, this will override any values specified for Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties. | | |
| | Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. | |
1 participant
https://issues.apache.org/jira/browse/MINIFICPP-2644
Depends on https://issues.apache.org/jira/browse/MINIFICPP-2644
Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:
Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
Has your PR been rebased against the latest commit within the target branch (typically main)?
Is your initial contribution a single, squashed commit?
For code changes:
For documentation related changes:
Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.