apollo_dashboard: add alert for dynamic config erroneous updates

[Itay-Tsabary-Starkware]

Note

Low Risk
Adds a new monitoring alert and minimal wiring; main risk is alert severity/threshold mismatch causing noisy paging.

Overview
Adds a new dashboard/Grafana alert config_manager_update_error_increase that fires when config_manager_update_errors increases over a 5m window.

Wires the alert into Apollo’s Rust alert registry via a new alert_scenarios::config_manager module and includes it in get_apollo_alerts, with severity set to Regular in code (and p2 in dev_grafana_alerts.json).

^{Written by Cursor Bugbot for commit 6ee838b. This will update automatically on new commits. Configure here.}

[reviewable-StarkWare]

This change is 

[Itay-Tsabary-Starkware]

• apollo_dashboard: add alert for dynamic config erroneous updates #12691 👈 (View in Graphite)
• apollo_config_manager: add update error metric #12687
• apollo_config_manager: unify error logging #12650
• main-v0.14.2

This stack of pull requests is managed by Graphite. Learn more about stacking.

[nadin-Starkware]

@nadin-Starkware reviewed 6 files and all commit messages, and made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on Itay-Tsabary-Starkware).

---

crates/apollo_dashboard/src/alert_scenarios/config_manager.rs line 18 at r2 (raw file):

/// Alert when the config manager reports update errors (load/validate or set dynamic config).
/// Uses default trigger timing: condition must hold for 30s before firing, evaluated every 30s.

Do you mean PENDING_DURATION_DEFAULT? The default value may change

Code quote:

30s

[Itay-Tsabary-Starkware]

@Itay-Tsabary-Starkware made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on nadin-Starkware).

---

crates/apollo_dashboard/src/alert_scenarios/config_manager.rs line 18 at r2 (raw file):

Previously, nadin-Starkware (Nadin Jbara) wrote…

Do you mean PENDING_DURATION_DEFAULT? The default value may change

Done (removed other verbose text).

[nadin-Starkware]

@nadin-Starkware reviewed 1 file and all commit messages, and resolved 1 discussion.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on Itay-Tsabary-Starkware).

[graphite-app]

Merge activity

• Feb 22, 7:34 AM UTC: This pull request can not be added to the Graphite merge queue. Please try rebasing and resubmitting to merge when ready.
• Feb 22, 7:34 AM UTC: Graphite disabled "merge when ready" on this PR due to: a merge conflict with the target branch; resolve the conflict and try again..

[Itay-Tsabary-Starkware]

@Itay-Tsabary-Starkware reviewed 9 files and all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on Itay-Tsabary-Starkware).