Skip to content

[WIP] Adding unit tests for the extend recommended alerts #2213

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
JianLi-RH wants to merge 1 commit into
base: main
Choose a base branch
from
+381 −0
Open

[WIP] Adding unit tests for the extend recommended alerts #2213

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions pkg/cli/admin/upgrade/recommend/examples/4.22.0-extend-recommended-alert-alerts.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
{
"status": "success",
"data": {
"alerts": [
{
"labels": {
"alertname": "VirtHandlerDaemonSetRolloutFailing",
"channel": "stable-4.22",
"namespace": "openshift-cluster-version",
"severity": "warning",
"upstream": "https://amd64.origin.releases.ci.openshift.org/graph",
"openShiftUpdatePrecheck": "true"
},
"annotations": {
"description": "For more information refer to 'oc adm upgrade' or https://console-openshift-console.apps.ci-ln-plnvytb-76ef8.aws-2.ci.openshift.org/settings/cluster/.",
"summary": "Your upstream update recommendation service recommends you update your cluster."
},
"state": "firing",
"activeAt": "2025-09-09T16:59:17.077287821Z",
"value": "2e+00",
"partialResponseStrategy": "WARN"
},
{
"labels": {
"alertname": "VMCannotBeEvicted",
"container": "insights-operator",
"description": "GSS does not provide enterprise-level support for an OKD cluster",
"endpoint": "https",
"info_link": "https://console.redhat.com/openshift/insights/advisor/clusters/7a18f603-2a0c-4794-af53-85bdae9b8ea9?first=ccx_rules_ocp.external.rules.okd_cluster_unsupported%7COKD_CLUSTER_UNSUPPORTED",
"instance": "10.130.0.31:8443",
"job": "metrics",
"namespace": "openshift-insights",
"pod": "insights-operator-b8b5f97fc-kl8rr",
"service": "metrics",
"severity": "warning",
"total_risk": "Moderate",
"openShiftUpdatePrecheck": "false"
},
"annotations": {
"description": "Insights recommendation \"GSS does not provide enterprise-level support for an OKD cluster\" with total risk \"Moderate\" was detected on the cluster. More information is available at https://console.redhat.com/openshift/insights/advisor/clusters/7a18f603-2a0c-4794-af53-85bdae9b8ea9?first=ccx_rules_ocp.external.rules.okd_cluster_unsupported%7COKD_CLUSTER_UNSUPPORTED.",
"summary": "An Insights recommendation is active for this cluster."
},
"state": "firing",
"activeAt": "2025-09-09T16:52:41.445352906Z",
"value": "1e+00",
"partialResponseStrategy": "WARN"
},
{
"labels": {
"alertname": "TestAlert",
"name": "ci-ln-plnvytb-76ef8-42h8v-worker-us-east-1a-84pg2",
"namespace": "openshift-machine-api",
"severity": "warning",
"openShiftUpdatePrecheck": "true"
},
"annotations": {
"description": "The machine has been without a Running or Deleting phase for more than 60 minutes.\nThe machine may not have been provisioned properly from the infrastructure provider, or\nit might have issues with CertificateSigningRequests being approved.\n",
"summary": "machine ci-ln-plnvytb-76ef8-42h8v-worker-us-east-1a-84pg2 is in phase: "
},
"state": "firing",
"activeAt": "2025-09-09T16:49:56.763336056Z",
"value": "1.757435418e+09",
"partialResponseStrategy": "WARN"
}
]
}
}
91 changes: 91 additions & 0 deletions pkg/cli/admin/upgrade/recommend/examples/4.22.0-extend-recommended-alert-cv.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
apiVersion: config.openshift.io/v1
kind: ClusterVersion
metadata:
creationTimestamp: "2025-09-09T16:25:09Z"
generation: 3
name: version
resourceVersion: "31455"
uid: 31502537-d38e-4652-a5a6-18b0d767b354
spec:
channel: stable-4.22
clusterID: 7a18f603-2a0c-4794-af53-85bdae9b8ea9
upstream: https://amd64.origin.releases.ci.openshift.org/graph
status:
availableUpdates:
- image: registry.ci.openshift.org/ocp/release@sha256:64db99b4f820a919ba98e077f181f6b385a6883b88984a5acbebae0aef319561
version: 4.22.0-0.nightly-2026-02-26-092444
capabilities:
enabledCapabilities:
- Build
- CSISnapshot
- CloudControllerManager
- CloudCredential
- Console
- DeploymentConfig
- ImageRegistry
- Ingress
- Insights
- MachineAPI
- NodeTuning
- OperatorLifecycleManager
- OperatorLifecycleManagerV1
- Storage
- baremetal
- marketplace
- openshift-samples
knownCapabilities:
- Build
- CSISnapshot
- CloudControllerManager
- CloudCredential
- Console
- DeploymentConfig
- ImageRegistry
- Ingress
- Insights
- MachineAPI
- NodeTuning
- OperatorLifecycleManager
- OperatorLifecycleManagerV1
- Storage
- baremetal
- marketplace
- openshift-samples
conditions:
- lastTransitionTime: "2025-09-09T16:58:30Z"
status: "True"
type: RetrievedUpdates
- lastTransitionTime: "2025-09-09T16:25:36Z"
message: Capabilities match configured spec
reason: AsExpected
status: "False"
type: ImplicitlyEnabledCapabilities
- lastTransitionTime: "2025-09-09T16:25:36Z"
message: Payload loaded version="4.22.0-ec.3" image="quay.io/openshift-release-dev/ocp-release@sha256:58b98da1492b3f4af6129c4684b8e8cde4f2dc197e4b483bb6025971d59f92a5"
architecture="amd64"
reason: PayloadLoaded
status: "True"
type: ReleaseAccepted
- lastTransitionTime: "2025-09-09T16:55:08Z"
message: Done applying 4.22.0-ec.3
status: "True"
type: Available
- lastTransitionTime: "2025-09-09T16:55:08Z"
status: "False"
type: Failing
- lastTransitionTime: "2025-09-09T16:55:08Z"
message: Cluster version is 4.22.0-ec.3
status: "False"
type: Progressing
desired:
image: quay.io/openshift-release-dev/ocp-release@sha256:58b98da1492b3f4af6129c4684b8e8cde4f2dc197e4b483bb6025971d59f92a5
version: 4.22.0-ec.3
history:
- completionTime: "2025-09-09T16:55:08Z"
image: quay.io/openshift-release-dev/ocp-release@sha256:58b98da1492b3f4af6129c4684b8e8cde4f2dc197e4b483bb6025971d59f92a5
startedTime: "2025-09-09T16:25:36Z"
state: Completed
verified: false
version: 4.22.0-ec.3
observedGeneration: 1
versionHash: XI4117989oE=
28 changes: 28 additions & 0 deletions pkg/cli/admin/upgrade/recommend/examples/4.22.0-extend-recommended-alert.output
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
The following conditions found no cause for concern in updating this cluster to later releases: recommended/CriticalAlerts (AsExpected), recommended/NodeAlerts (AsExpected), recommended/PodDisruptionBudgetAlerts (AsExpected), recommended/PodImagePullAlerts (AsExpected)

The following conditions found cause for concern in updating this cluster to later releases: recommended/UpdatePrecheckAlerts/TestAlert/2, recommended/UpdatePrecheckAlerts/VirtHandlerDaemonSetRolloutFailing/0, recommended/VirtAlerts/VMCannotBeEvicted/1

recommended/UpdatePrecheckAlerts/VirtHandlerDaemonSetRolloutFailing/0=False:

Reason: Alert:firing
Message: warning alert VirtHandlerDaemonSetRolloutFailing firing, suggesting issues worth investigating before updating the cluster. Your upstream update recommendation service recommends you update your cluster. The alert description is: For more information refer to 'oc adm upgrade' or https://console-openshift-console.apps.ci-ln-plnvytb-76ef8.aws-2.ci.openshift.org/settings/cluster/. <alert does not have a runbook_url annotation>

recommended/VirtAlerts/VMCannotBeEvicted/1=False:

Reason: Alert:firing
Message: warning alert VMCannotBeEvicted firing, which may slow workload redistribution during rolling node updates. An Insights recommendation is active for this cluster. The alert description is: Insights recommendation "GSS does not provide enterprise-level support for an OKD cluster" with total risk "Moderate" was detected on the cluster. More information is available at https://console.redhat.com/openshift/insights/advisor/clusters/7a18f603-2a0c-4794-af53-85bdae9b8ea9?first=ccx_rules_ocp.external.rules.okd_cluster_unsupported%7COKD_CLUSTER_UNSUPPORTED. <alert does not have a runbook_url annotation>

recommended/UpdatePrecheckAlerts/TestAlert/2=False:

Reason: Alert:firing
Message: warning alert TestAlert firing, suggesting issues worth investigating before updating the cluster. machine ci-ln-plnvytb-76ef8-42h8v-worker-us-east-1a-84pg2 is in phase: . The alert description is: The machine has been without a Running or Deleting phase for more than 60 minutes.
The machine may not have been provisioned properly from the infrastructure provider, or
it might have issues with CertificateSigningRequests being approved.
<alert does not have a runbook_url annotation>

Upstream update service: https://amd64.origin.releases.ci.openshift.org/graph
Channel: stable-4.22

Updates to 4.22:
VERSION ISSUES
4.22.0-0.nightly-2026-02-26-092444 no known issues relevant to this cluster
28 changes: 28 additions & 0 deletions .../upgrade/recommend/examples/4.22.0-extend-recommended-alert.show-outdated-releases-output
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
The following conditions found no cause for concern in updating this cluster to later releases: recommended/CriticalAlerts (AsExpected), recommended/NodeAlerts (AsExpected), recommended/PodDisruptionBudgetAlerts (AsExpected), recommended/PodImagePullAlerts (AsExpected)

The following conditions found cause for concern in updating this cluster to later releases: recommended/UpdatePrecheckAlerts/TestAlert/2, recommended/UpdatePrecheckAlerts/VirtHandlerDaemonSetRolloutFailing/0, recommended/VirtAlerts/VMCannotBeEvicted/1

recommended/UpdatePrecheckAlerts/VirtHandlerDaemonSetRolloutFailing/0=False:

Reason: Alert:firing
Message: warning alert VirtHandlerDaemonSetRolloutFailing firing, suggesting issues worth investigating before updating the cluster. Your upstream update recommendation service recommends you update your cluster. The alert description is: For more information refer to 'oc adm upgrade' or https://console-openshift-console.apps.ci-ln-plnvytb-76ef8.aws-2.ci.openshift.org/settings/cluster/. <alert does not have a runbook_url annotation>

recommended/VirtAlerts/VMCannotBeEvicted/1=False:

Reason: Alert:firing
Message: warning alert VMCannotBeEvicted firing, which may slow workload redistribution during rolling node updates. An Insights recommendation is active for this cluster. The alert description is: Insights recommendation "GSS does not provide enterprise-level support for an OKD cluster" with total risk "Moderate" was detected on the cluster. More information is available at https://console.redhat.com/openshift/insights/advisor/clusters/7a18f603-2a0c-4794-af53-85bdae9b8ea9?first=ccx_rules_ocp.external.rules.okd_cluster_unsupported%7COKD_CLUSTER_UNSUPPORTED. <alert does not have a runbook_url annotation>

recommended/UpdatePrecheckAlerts/TestAlert/2=False:

Reason: Alert:firing
Message: warning alert TestAlert firing, suggesting issues worth investigating before updating the cluster. machine ci-ln-plnvytb-76ef8-42h8v-worker-us-east-1a-84pg2 is in phase: . The alert description is: The machine has been without a Running or Deleting phase for more than 60 minutes.
The machine may not have been provisioned properly from the infrastructure provider, or
it might have issues with CertificateSigningRequests being approved.
<alert does not have a runbook_url annotation>
Comment on lines +18 to +21
Copy link

@coderabbitai coderabbitai bot Feb 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Normalize wrapped-message indentation in the golden output.

Line 21 has an extra leading space versus Line 19 and Line 20, making this block inconsistently formatted.

Suggested fix 
-   <alert does not have a runbook_url annotation>
+  <alert does not have a runbook_url annotation>
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
Message: warning alert TestAlert firing, suggesting issues worth investigating before updating the cluster. machine ci-ln-plnvytb-76ef8-42h8v-worker-us-east-1a-84pg2 is in phase: . The alert description is: The machine has been without a Running or Deleting phase for more than 60 minutes.
The machine may not have been provisioned properly from the infrastructure provider, or
it might have issues with CertificateSigningRequests being approved.
<alert does not have a runbook_url annotation>
Message: warning alert TestAlert firing, suggesting issues worth investigating before updating the cluster. machine ci-ln-plnvytb-76ef8-42h8v-worker-us-east-1a-84pg2 is in phase: . The alert description is: The machine has been without a Running or Deleting phase for more than 60 minutes.
The machine may not have been provisioned properly from the infrastructure provider, or
it might have issues with CertificateSigningRequests being approved.
<alert does not have a runbook_url annotation>
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In
`@pkg/cli/admin/upgrade/recommend/examples/4.22.0-extend-recommended-alert.show-outdated-releases-output`
around lines 18 - 21, The wrapped "Message:" block has inconsistent indentation
(an extra leading space on the third wrapped line); edit the golden output so
that the third line of the wrapped message matches the indentation of the
previous wrapped lines (remove the extra leading space before "it might have
issues..." in the Message block) to normalize formatting and ensure all wrapped
lines under "Message:" align consistently.


Upstream update service: https://amd64.origin.releases.ci.openshift.org/graph
Channel: stable-4.22

Updates to 4.22:
VERSION ISSUES
4.22.0-0.nightly-2026-02-26-092444 no known issues relevant to this cluster
42 changes: 42 additions & 0 deletions ...cli/admin/upgrade/recommend/examples/4.22.0-extend-recommended-critical-alert-alerts.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
{
"status": "success",
"data": {
"alerts": [
{
"labels": {
"alertname": "VirtHandlerDaemonSetRolloutFailing",
"channel": "stable-4.22",
"namespace": "openshift-cluster-version",
"severity": "critical",
"upstream": "https://amd64.origin.releases.ci.openshift.org/graph",
"openShiftUpdatePrecheck": "true"
},
"annotations": {
"description": "For more information refer to 'oc adm upgrade' or https://console-openshift-console.apps.ci-ln-plnvytb-76ef8.aws-2.ci.openshift.org/settings/cluster/.",
"summary": "Your upstream update recommendation service recommends you update your cluster."
},
"state": "firing",
"activeAt": "2025-09-09T16:59:17.077287821Z",
"value": "2e+00",
"partialResponseStrategy": "WARN"
},
{
"labels": {
"alertname": "VMCannotBeEvicted",
"name": "ci-ln-plnvytb-76ef8-42h8v-worker-us-east-1a-84pg2",
"namespace": "openshift-machine-api",
"severity": "critical",
"openShiftUpdatePrecheck": "true"
},
"annotations": {
"description": "The machine has been without a Running or Deleting phase for more than 60 minutes.\nThe machine may not have been provisioned properly from the infrastructure provider, or\nit might have issues with CertificateSigningRequests being approved.\n",
"summary": "machine ci-ln-plnvytb-76ef8-42h8v-worker-us-east-1a-84pg2 is in phase: "
},
"state": "pending",
"activeAt": "2025-09-09T16:49:56.763336056Z",
"value": "1.757435418e+09",
"partialResponseStrategy": "WARN"
}
]
}
}
91 changes: 91 additions & 0 deletions pkg/cli/admin/upgrade/recommend/examples/4.22.0-extend-recommended-critical-alert-cv.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
apiVersion: config.openshift.io/v1
kind: ClusterVersion
metadata:
creationTimestamp: "2025-09-09T16:25:09Z"
generation: 3
name: version
resourceVersion: "31455"
uid: 31502537-d38e-4652-a5a6-18b0d767b354
spec:
channel: stable-4.22
clusterID: 7a18f603-2a0c-4794-af53-85bdae9b8ea9
upstream: https://amd64.origin.releases.ci.openshift.org/graph
status:
availableUpdates:
- image: registry.ci.openshift.org/ocp/release@sha256:64db99b4f820a919ba98e077f181f6b385a6883b88984a5acbebae0aef319561
version: 4.22.0-0.nightly-2026-02-26-092444
capabilities:
enabledCapabilities:
- Build
- CSISnapshot
- CloudControllerManager
- CloudCredential
- Console
- DeploymentConfig
- ImageRegistry
- Ingress
- Insights
- MachineAPI
- NodeTuning
- OperatorLifecycleManager
- OperatorLifecycleManagerV1
- Storage
- baremetal
- marketplace
- openshift-samples
knownCapabilities:
- Build
- CSISnapshot
- CloudControllerManager
- CloudCredential
- Console
- DeploymentConfig
- ImageRegistry
- Ingress
- Insights
- MachineAPI
- NodeTuning
- OperatorLifecycleManager
- OperatorLifecycleManagerV1
- Storage
- baremetal
- marketplace
- openshift-samples
conditions:
- lastTransitionTime: "2025-09-09T16:58:30Z"
status: "True"
type: RetrievedUpdates
- lastTransitionTime: "2025-09-09T16:25:36Z"
message: Capabilities match configured spec
reason: AsExpected
status: "False"
type: ImplicitlyEnabledCapabilities
- lastTransitionTime: "2025-09-09T16:25:36Z"
message: Payload loaded version="4.22.0-ec.3" image="quay.io/openshift-release-dev/ocp-release@sha256:58b98da1492b3f4af6129c4684b8e8cde4f2dc197e4b483bb6025971d59f92a5"
architecture="amd64"
reason: PayloadLoaded
status: "True"
type: ReleaseAccepted
- lastTransitionTime: "2025-09-09T16:55:08Z"
message: Done applying 4.22.0-ec.3
status: "True"
type: Available
- lastTransitionTime: "2025-09-09T16:55:08Z"
status: "False"
type: Failing
- lastTransitionTime: "2025-09-09T16:55:08Z"
message: Cluster version is 4.22.0-ec.3
status: "False"
type: Progressing
desired:
image: quay.io/openshift-release-dev/ocp-release@sha256:58b98da1492b3f4af6129c4684b8e8cde4f2dc197e4b483bb6025971d59f92a5
version: 4.22.0-ec.3
history:
- completionTime: "2025-09-09T16:55:08Z"
image: quay.io/openshift-release-dev/ocp-release@sha256:58b98da1492b3f4af6129c4684b8e8cde4f2dc197e4b483bb6025971d59f92a5
startedTime: "2025-09-09T16:25:36Z"
state: Completed
verified: false
version: 4.22.0-ec.3
observedGeneration: 1
versionHash: XI4117989oE=
15 changes: 15 additions & 0 deletions pkg/cli/admin/upgrade/recommend/examples/4.22.0-extend-recommended-critical-alert.output
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
The following conditions found no cause for concern in updating this cluster to later releases: recommended/NodeAlerts (AsExpected), recommended/PodDisruptionBudgetAlerts (AsExpected), recommended/PodImagePullAlerts (AsExpected), recommended/UpdatePrecheckAlerts (AsExpected)

The following conditions found cause for concern in updating this cluster to later releases: recommended/CriticalAlerts/VirtHandlerDaemonSetRolloutFailing/0

recommended/CriticalAlerts/VirtHandlerDaemonSetRolloutFailing/0=False:

Reason: Alert:firing
Message: critical alert VirtHandlerDaemonSetRolloutFailing firing, suggesting significant cluster issues worth investigating. Your upstream update recommendation service recommends you update your cluster. The alert description is: For more information refer to 'oc adm upgrade' or https://console-openshift-console.apps.ci-ln-plnvytb-76ef8.aws-2.ci.openshift.org/settings/cluster/. <alert does not have a runbook_url annotation>

Upstream update service: https://amd64.origin.releases.ci.openshift.org/graph
Channel: stable-4.22

Updates to 4.22:
VERSION ISSUES
4.22.0-0.nightly-2026-02-26-092444 no known issues relevant to this cluster
15 changes: 15 additions & 0 deletions ...recommend/examples/4.22.0-extend-recommended-critical-alert.show-outdated-releases-output
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
The following conditions found no cause for concern in updating this cluster to later releases: recommended/NodeAlerts (AsExpected), recommended/PodDisruptionBudgetAlerts (AsExpected), recommended/PodImagePullAlerts (AsExpected), recommended/UpdatePrecheckAlerts (AsExpected)

The following conditions found cause for concern in updating this cluster to later releases: recommended/CriticalAlerts/VirtHandlerDaemonSetRolloutFailing/0

recommended/CriticalAlerts/VirtHandlerDaemonSetRolloutFailing/0=False:

Reason: Alert:firing
Message: critical alert VirtHandlerDaemonSetRolloutFailing firing, suggesting significant cluster issues worth investigating. Your upstream update recommendation service recommends you update your cluster. The alert description is: For more information refer to 'oc adm upgrade' or https://console-openshift-console.apps.ci-ln-plnvytb-76ef8.aws-2.ci.openshift.org/settings/cluster/. <alert does not have a runbook_url annotation>

Upstream update service: https://amd64.origin.releases.ci.openshift.org/graph
Channel: stable-4.22

Updates to 4.22:
VERSION ISSUES
4.22.0-0.nightly-2026-02-26-092444 no known issues relevant to this cluster
4 changes: 4 additions & 0 deletions pkg/cli/admin/upgrade/recommend/examples_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,10 @@ func TestExamples(t *testing.T) {
var version string
if version = variant.versions[cv]; version != "" {
variant.outputSuffix = fmt.Sprintf(variant.outputSuffixPattern, version)
} else {
if variant.name == "specific version" {
continue
}
}
t.Run(fmt.Sprintf("%s-%s", cv, variant.name), func(t *testing.T) {
t.Parallel()
Expand Down