Skip to content

Add support for drift detection ignore rules #1627

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
dipti-pai wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Add support for drift detection ignore rules #1627

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions api/v1/kustomization_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,13 @@ type KustomizationSpec struct {
// The expressions are evaluated only when Wait or HealthChecks are specified.
// +optional
HealthCheckExprs []kustomize.CustomHealthCheck `json:"healthCheckExprs,omitempty"`

// DriftIgnoreRules is a list of rules for specifying which changes to ignore
// during drift detection. These rules are applied to the resources managed
// by the Kustomization and are used to exclude specific JSON pointer paths
// from the drift detection and apply process.
// +optional
DriftIgnoreRules []DriftIgnoreRule `json:"driftIgnoreRules,omitempty"`
}

// CommonMetadata defines the common labels and annotations.
Expand All @@ -205,6 +212,22 @@ type CommonMetadata struct {
Labels map[string]string `json:"labels,omitempty"`
}

// DriftIgnoreRule defines a rule to selectively disregard specific changes during
// the drift detection process.
type DriftIgnoreRule struct {
// Paths is a list of JSON Pointer (RFC 6901) paths to be excluded from
// consideration in a Kubernetes object.
// +required
Paths []string `json:"paths"`

// Target is a selector for specifying Kubernetes objects to which this
// rule applies.
// If Target is not set, the Paths will be ignored for all Kubernetes
// objects within the manifest of the Kustomization.
// +optional
Target *kustomize.Selector `json:"target,omitempty"`
}

// Decryption defines how decryption is handled for Kubernetes manifests.
type Decryption struct {
// Provider is the name of the decryption engine.
Expand Down
32 changes: 32 additions & 0 deletions api/v1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

67 changes: 67 additions & 0 deletions config/crd/bases/kustomize.toolkit.fluxcd.io_kustomizations.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,73 @@ spec:
- name
type: object
type: array
driftIgnoreRules:
description: |-
DriftIgnoreRules is a list of rules for specifying which changes to ignore
during drift detection. These rules are applied to the resources managed
by the Kustomization and are used to exclude specific JSON pointer paths
from the drift detection and apply process.
items:
description: |-
DriftIgnoreRule defines a rule to selectively disregard specific changes during
the drift detection process.
properties:
paths:
description: |-
Paths is a list of JSON Pointer (RFC 6901) paths to be excluded from
consideration in a Kubernetes object.
items:
type: string
type: array
target:
description: |-
Target is a selector for specifying Kubernetes objects to which this
rule applies.
If Target is not set, the Paths will be ignored for all Kubernetes
objects within the manifest of the Kustomization.
properties:
annotationSelector:
description: |-
AnnotationSelector is a string that follows the label selection expression
https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#api
It matches with the resource annotations.
type: string
group:
description: |-
Group is the API group to select resources from.
Together with Version and Kind it is capable of unambiguously identifying and/or selecting resources.
https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/api-group.md
type: string
kind:
description: |-
Kind of the API Group to select resources from.
Together with Group and Version it is capable of unambiguously
identifying and/or selecting resources.
https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/api-group.md
type: string
labelSelector:
description: |-
LabelSelector is a string that follows the label selection expression
https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#api
It matches with the resource labels.
type: string
name:
description: Name to match resources with.
type: string
namespace:
description: Namespace to select resources from.
type: string
version:
description: |-
Version of the API Group to select resources from.
Together with Group and Kind it is capable of unambiguously identifying and/or selecting resources.
https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/api-group.md
type: string
type: object
required:
- paths
type: object
type: array
force:
default: false
description: |-
Expand Down
85 changes: 85 additions & 0 deletions docs/api/v1/kustomize.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -425,6 +425,23 @@ health of custom resources using Common Expression Language (CEL).
The expressions are evaluated only when Wait or HealthChecks are specified.</p>
</td>
</tr>
<tr>
<td>
<code>driftIgnoreRules</code><br>
<em>
<a href="#kustomize.toolkit.fluxcd.io/v1.DriftIgnoreRule">
[]DriftIgnoreRule
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>DriftIgnoreRules is a list of rules for specifying which changes to ignore
during drift detection. These rules are applied to the resources managed
by the Kustomization and are used to exclude specific JSON pointer paths
from the drift detection and apply process.</p>
</td>
</tr>
</table>
</td>
</tr>
Expand Down Expand Up @@ -684,6 +701,57 @@ the feature gate <code>AdditiveCELDependencyCheck</code> must be set to <code>tr
</table>
</div>
</div>
<h3 id="kustomize.toolkit.fluxcd.io/v1.DriftIgnoreRule">DriftIgnoreRule
</h3>
<p>
(<em>Appears on:</em>
<a href="#kustomize.toolkit.fluxcd.io/v1.KustomizationSpec">KustomizationSpec</a>)
</p>
<p>DriftIgnoreRule defines a rule to selectively disregard specific changes during
the drift detection process.</p>
<div class="md-typeset__scrollwrap">
<div class="md-typeset__table">
<table>
<thead>
<tr>
<th>Field</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<code>paths</code><br>
<em>
[]string
</em>
</td>
<td>
<p>Paths is a list of JSON Pointer (RFC 6901) paths to be excluded from
consideration in a Kubernetes object.</p>
</td>
</tr>
<tr>
<td>
<code>target</code><br>
<em>
<a href="https://godoc.org/github.com/fluxcd/pkg/apis/kustomize#Selector">
github.com/fluxcd/pkg/apis/kustomize.Selector
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>Target is a selector for specifying Kubernetes objects to which this
rule applies.
If Target is not set, the Paths will be ignored for all Kubernetes
objects within the manifest of the Kustomization.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<h3 id="kustomize.toolkit.fluxcd.io/v1.KustomizationSpec">KustomizationSpec
</h3>
<p>
Expand Down Expand Up @@ -1058,6 +1126,23 @@ health of custom resources using Common Expression Language (CEL).
The expressions are evaluated only when Wait or HealthChecks are specified.</p>
</td>
</tr>
<tr>
<td>
<code>driftIgnoreRules</code><br>
<em>
<a href="#kustomize.toolkit.fluxcd.io/v1.DriftIgnoreRule">
[]DriftIgnoreRule
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>DriftIgnoreRules is a list of rules for specifying which changes to ignore
during drift detection. These rules are applied to the resources managed
by the Kustomization and are used to exclude specific JSON pointer paths
from the drift detection and apply process.</p>
</td>
</tr>
</tbody>
</table>
</div>
Expand Down
114 changes: 114 additions & 0 deletions docs/spec/v1/kustomizations.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -854,6 +854,120 @@ kustomize.toolkit.fluxcd.io/force: enabled
This way, only the targeted resources are force-replaced when immutable field
changes are made. The annotation should be removed after the change is applied.

### Drift Ignore Rules

`.spec.driftIgnoreRules` is an optional list used to selectively ignore changes
to specific fields during drift detection and correction. This allows external
controllers or tools to manage certain fields on Kubernetes resources without
having those changes reverted by the kustomize-controller during reconciliation.

Each item in the list must have the following fields:

- `paths` (required): A list of [JSON Pointer (RFC 6901)](https://datatracker.ietf.org/doc/html/rfc6901)
paths to exclude from drift detection. These paths refer to specific fields
within the Kubernetes object manifest.
- `target` (optional): A selector to scope the rule to specific Kubernetes
resources. If not set, the paths are ignored for all resources in the
Kustomization.

**Warning:** Omitting the `target` selector causes the rule to match **all**
objects managed by the Kustomization. Always scope rules to specific resources
using `target` unless you intentionally want to ignore the specified paths
across every resource.

The `target` selector supports the following fields:

| Field | Description |
|----------------------|--------------------------------------|
| `group` | API group (regex) |
| `version` | API version (regex) |
| `kind` | Resource kind (regex) |
| `name` | Resource name (regex) |
| `namespace` | Resource namespace (regex) |
| `labelSelector` | Kubernetes label selector expression |
| `annotationSelector` | Kubernetes annotation selector expression |

**Note:** The `group`, `version`, `kind`, `name`, and `namespace` fields
support regex patterns. The `labelSelector` and `annotationSelector` fields
use the standard Kubernetes
[label selector](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors)
syntax.

**Note:** For JSON Pointer paths that contain `/` in the key name (e.g.
annotation keys), the `/` must be escaped as `~1` per
[RFC 6901](https://datatracker.ietf.org/doc/html/rfc6901#section-3).
For example, the annotation `external-dns.alpha.kubernetes.io/hostname`
would be referenced as `/metadata/annotations/external-dns.alpha.kubernetes.io~1hostname`.

#### Ignore a field on all resources

To ignore a specific field on all resources managed by the Kustomization:

```yaml
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: app
namespace: flux-system
spec:
# ...omitted for brevity
driftIgnoreRules:
- paths:
- "/spec/replicas"
```

#### Ignore fields on specific resources

To ignore fields only on resources that match a target selector:

```yaml
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: app
namespace: flux-system
spec:
# ...omitted for brevity
driftIgnoreRules:
- paths:
- "/spec/replicas"
target:
kind: Deployment
- paths:
- "/metadata/annotations/external-dns.alpha.kubernetes.io~1hostname"
target:
kind: Service
name: my-service
- paths:
- "/spec/template/spec/containers/0/resources"
target:
kind: Deployment
name: my-app
```

In the above example:

- The `/spec/replicas` field is ignored on all Deployments, allowing
an HPA or other autoscaler to manage the replica count without
interference from the kustomize-controller.
- The `external-dns.alpha.kubernetes.io/hostname` annotation is ignored
on a specific Service named `my-service`, allowing external-dns to
manage this annotation.
- The entire `/resources` subtree under container spec is ignored on
a specific Deployment named `my-app`, allowing a VPA or other resource
management tool to adjust container resources.

**Important:** Drift ignore rules work with the
[server-side apply](https://kubernetes.io/docs/reference/using-api/server-side-apply/)
field ownership model. The ignored paths are removed from the desired state
before the controller applies, which relinquishes the controller's ownership of
those fields. For the ignored fields to be preserved, they must be owned by
another field manager (e.g. another controller or a `kubectl apply --server-side
--field-manager=<name>` invocation). If no other field manager owns the field,
the API server may remove it during garbage collection of abandoned fields.

### KubeConfig (Remote clusters)

With the `.spec.kubeConfig` field a Kustomization
Expand Down
Loading
Loading