The grafana.ini.j2 template sets root_url to a relative path (/ui/). This causes Grafana to generate relative URLs in alert notification payloads; for example, embed URLs like "/ui/alerting/list" instead of fully qualified URLs.
Discord's webhook API rejects payloads containing invalid URLs in embed fields, returning:
webhook failed validation: the Discord API responded (status 400) with error code 0
Expected Behavior
root_url should be a full URL (e.g. https://g.pigsty/ui/) so that Grafana generates valid absolute URLs in all outgoing notifications.
Workaround
Manually edit /etc/grafana/grafana.ini after deployment:
[server]
root_url = https://your-grafana-domain/ui/
Then restart Grafana. Note this gets overwritten on the next ./infra.yml -t grafana run.
As a permanent fix, I guess we could allow one to configure this through the pigsty.yml file
- Expose a
pigsty.yml variable (e.g. grafana_root_url) that gets templated into grafana.ini.j2, defaulting to http://{{ admin_ip }}/ui/ or deriving from the infra_portal.grafana.domain definition.
- Update the template to build
root_url from existing variables like infra_portal.grafana.domain and infra_portal.grafana.endpoint.
The
grafana.ini.j2template setsroot_urlto a relative path (/ui/). This causes Grafana to generate relative URLs in alert notification payloads; for example, embed URLs like"/ui/alerting/list"instead of fully qualified URLs.Discord's webhook API rejects payloads containing invalid URLs in embed fields, returning:
Expected Behavior
root_urlshould be a full URL (e.g.https://g.pigsty/ui/) so that Grafana generates valid absolute URLs in all outgoing notifications.Workaround
Manually edit
/etc/grafana/grafana.iniafter deployment:Then restart Grafana. Note this gets overwritten on the next
./infra.yml -t grafanarun.As a permanent fix, I guess we could allow one to configure this through the pigsty.yml file
pigsty.ymlvariable (e.g.grafana_root_url) that gets templated intografana.ini.j2, defaulting tohttp://{{ admin_ip }}/ui/or deriving from theinfra_portal.grafana.domaindefinition.root_urlfrom existing variables likeinfra_portal.grafana.domainandinfra_portal.grafana.endpoint.