[+] implement admin.drop_source_partitions for proactive decommission cleanup#1364
Open
RafayKhattak wants to merge 1 commit intocybertec-postgresql:masterfrom
Open
[+] implement admin.drop_source_partitions for proactive decommission cleanup#1364RafayKhattak wants to merge 1 commit intocybertec-postgresql:masterfrom
RafayKhattak wants to merge 1 commit intocybertec-postgresql:masterfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1340
When you decommission a monitored database, its metric data stays behind in the
partitions until retention kicks in. If retention is set to something like 90 days,
that's a lot of dead data sitting around. The only other option right now is manually
hunting down and dropping the partition tables yourself, which is tedious because
a single source can have dozens of partitions scattered across every metric.
This adds a single function you can call:
It finds all Level-2 (dbname) partitions belonging to that source by reading the
partition bound expressions from
pg_catalog, so it works correctly even when thetable name uses the MD5 hash fallback for long database names. For each one it
acquires the same advisory lock used by
ensure_partition_metric_dbname_time,detaches and drops the partition (which cascades to all the time sub-partitions),
then cleans up
admin.all_distinct_dbname_metricsso the source disappears fromGrafana dropdowns.
The function is idempotent, calling it twice or for a source that doesn't exist
just returns 0.
Changes
internal/sinks/sql/drop_source_partitions.sqlinternal/sinks/postgres.go(embed + add to schema init slice)Testing
Tested against a live multi-tenant Docker environment with both normal-length and
63+ character database names (MD5 hash path).