feat(controllers): Enable multiple controllers with different label selectors for the same entity type (V2)#1070
Conversation
…electors for the same entity type
@dotnet-policy-service agree |
There was a problem hiding this comment.
Pull request overview
Adds support for registering multiple IEntityController<TEntity> implementations for the same entity type and dispatching reconciliation/deletion events to all controllers whose label selector matches the entity’s labels.
Changes:
- Adds
LabelFilteras a defaulted property onIEntityController<TEntity>to allow per-controller label scoping. - Updates DI registration to allow multiple scoped controller registrations for the same entity type.
- Introduces runtime dispatch in
Reconciler<TEntity>and aLabelSelectorMatcherutility with accompanying unit tests.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| test/KubeOps.Operator.Test/Reconciliation/Reconciler.Test.cs | Updates existing reconciler tests to mock resolving IEnumerable<IEntityController<T>>. |
| test/KubeOps.Operator.Test/Reconciliation/Reconciler.MultiController.Test.cs | Adds tests covering multi-controller dispatch, ordering, and short-circuit behavior. |
| test/KubeOps.Operator.Test/Reconciliation/LabelSelectorMatcher.Test.cs | Adds unit tests for selector parsing/matching behavior. |
| test/KubeOps.Operator.Test/Builder/OperatorBuilder.Test.cs | Verifies multiple controller registrations and that watchers aren’t duplicated. |
| src/KubeOps.Operator/Reconciliation/Reconciler.cs | Replaces single-controller resolution with dispatch to all matching controllers. |
| src/KubeOps.Operator/Reconciliation/LabelSelectorMatcher.cs | Adds selector-expression matcher for KubeOps’ set-based selector syntax. |
| src/KubeOps.Operator/Builder/OperatorBuilder.cs | Switches controller DI registration from TryAddScoped to AddScoped to allow multiples. |
| src/KubeOps.Abstractions/Reconciliation/Controller/IEntityController{TEntity}.cs | Adds LabelFilter default interface property and documentation. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| internal static bool Matches(string? selector, IReadOnlyDictionary<string, string>? entityLabels) | ||
| { | ||
| if (selector is null) return true; | ||
| entityLabels ??= new Dictionary<string, string>(); | ||
|
|
||
| return SplitTopLevel(selector).All(clause => MatchClause(clause, entityLabels)); | ||
| } |
There was a problem hiding this comment.
Matches treats only null as a catch-all, but an empty/whitespace selector string currently matches nothing (because it becomes an empty clause). LabelSelector[].ToExpression() returns an empty string for an empty selector list, and Kubernetes semantics for an empty selector are "match all". Consider handling string.IsNullOrWhiteSpace(selector) as true and/or skipping empty clauses in SplitTopLevel (also covers trailing commas). Add a unit test for the empty-string case to prevent regressions.
| /// Supports the set-based formats emitted by KubeOps.KubernetesClient label-selector types: | ||
| /// <list type="bullet"> | ||
| /// <item><c>key in (v1,v2)</c> – EqualsSelector</item> | ||
| /// <item><c>key notin (v1,v2)</c> – NotEqualsSelector</item> | ||
| /// <item><c>key</c> – ExistsSelector</item> | ||
| /// <item><c>!key</c> – NotExistsSelector</item> |
There was a problem hiding this comment.
The XML doc says this matcher "supports the set-based formats emitted" by the KubeOps label-selector types, but the implementation also supports key=value and key!=value clauses. Either document these additional supported operators or remove them to keep behavior and docs aligned.
| /// Supports the set-based formats emitted by KubeOps.KubernetesClient label-selector types: | |
| /// <list type="bullet"> | |
| /// <item><c>key in (v1,v2)</c> – EqualsSelector</item> | |
| /// <item><c>key notin (v1,v2)</c> – NotEqualsSelector</item> | |
| /// <item><c>key</c> – ExistsSelector</item> | |
| /// <item><c>!key</c> – NotExistsSelector</item> | |
| /// Supports the set-based formats emitted by KubeOps.KubernetesClient label-selector types, as well as equality-based clauses: | |
| /// <list type="bullet"> | |
| /// <item><c>key in (v1,v2)</c> – EqualsSelector</item> | |
| /// <item><c>key notin (v1,v2)</c> – NotEqualsSelector</item> | |
| /// <item><c>key</c> – ExistsSelector</item> | |
| /// <item><c>!key</c> – NotExistsSelector</item> | |
| /// <item><c>key=value</c> – equality match</item> | |
| /// <item><c>key!=value</c> – inequality match</item> |
| if (controllers.Count == 0) | ||
| { | ||
| logger.LogWarning( | ||
| """No controller matched labels for "{Kind}/{Name}". Skipping.""", | ||
| entity.Kind, | ||
| entity.Name()); | ||
| return ReconciliationResult<TEntity>.Success(entity); |
There was a problem hiding this comment.
Logging a Warning for every event where no controller matches can create very noisy logs in the intended use-case (multiple scoped controllers, no catch-all). Since "no match" can be an expected outcome, consider lowering this to Information/Debug, or otherwise rate-limiting/aggregating it (e.g., include a counter, log once per kind, or gate behind a setting).
| /// | ||
| /// When multiple controllers are registered for the same entity type the reconciler evaluates every | ||
| /// controller's <see cref="LabelFilter"/> against the entity's labels and dispatches to all that | ||
| /// match, allowing fine-grained fan-out without touching the watcher or DI registration plumbing. |
There was a problem hiding this comment.
The doc comment mentions fan-out works "without touching the watcher or DI registration plumbing", but enabling multiple controllers for the same entity type does require DI registration changes (e.g., TryAddScoped -> AddScoped). Reword this to avoid misleading readers (e.g., emphasize no watcher changes / no signature changes instead).
| /// match, allowing fine-grained fan-out without touching the watcher or DI registration plumbing. | |
| /// match, allowing fine-grained fan-out without changing the watcher logic or controller method signatures. |
| public IOperatorBuilder AddController<TImplementation, TEntity>() | ||
| where TImplementation : class, IEntityController<TEntity> | ||
| where TEntity : IKubernetesObject<V1ObjectMeta> | ||
| { | ||
| Services.TryAddScoped<IEntityController<TEntity>, TImplementation>(); | ||
| Services.AddScoped<IEntityController<TEntity>, TImplementation>(); | ||
| Services.TryAddSingleton<IReconciler<TEntity>, Reconciler<TEntity>>(); |
There was a problem hiding this comment.
Switching to AddScoped enables multiple IEntityController<TEntity> registrations, but note that ResourceWatcher<TEntity> still uses a single IEntityLabelSelector<TEntity> for server-side watch filtering. If users register multiple controllers via AddController<TImplementation,TEntity,TLabelSelector>() with different selectors, only the first selector is used (due to TryAddSingleton), which can prevent events for other controllers from ever reaching the reconciler. Consider documenting this limitation clearly and/or failing fast when multiple different label selectors are registered for the same entity type.
|
Hey @stevefan1999-personal thanks for your contribution and sorry for the late reply. In general I like the idea of having multiple controller/finalizer registrations and dispatching events to those being responsible. Based on requests from the past there is just one thing I would like to change. This pull request covers a single - most probably your - use-case. dispatching based on labels. we had some requests in the past like #803 or to reconcile based on status updates (not by generation - sorry couldn't actually find the ticket by now, might be in discord). If we could implement this feature a bit more generic/flexibel that would help in resolving those requests as well. I would like to see something like:
ValueTask<bool> ShouldHandle(TEntity entity)
=> ValueTask.FromResult(true);
foreach (var controller in controllers)
{
if (await controller.ShouldHandle(entity))
{
return await controller.ReconcileAsync(entity, cancellationToken);
}
}
return ReconciliationResult<TEntity>.Failure(entity, "No responsible controller found.");this reduces the specific use-case complexity inside the resource-watcher and give more opportunities/flexibilities to the consumers. what do you think? |
|
replaced by #1084 |
Multi-controller dispatch via
LabelFilteronIEntityController<TEntity>Fixes #909. A continuation of #911
Overview
Adds support for registering multiple controllers for the same Kubernetes entity type, each handling a different subset of resources based on label selectors. The implementation follows the approach requested in review: a single property on the existing interface, runtime dispatch, and no breaking changes.
Changes
IEntityController<TEntity>— one new default propertyControllers that need to scope themselves declare a selector expression:
OperatorBuilder—AddScopedinstead ofTryAddScopedOne line change.
TryAddScopedsilently drops duplicate service registrations;AddScopedappends them, enablingGetServices<IEntityController<TEntity>>()to resolve all registered controllers for an entity type.Reconciler<TEntity>— dispatch to all matching controllersNew
DispatchToMatchingControllersmethod replaces the singleGetRequiredServicecall. It resolves all registered controllers, filters byLabelFilteragainst the entity's labels, and calls each matching one in registration order. On the first failure the chain short-circuits.LabelSelectorMatcher— new internal utilityEvaluates KubeOps set-based selector expressions (
key in (v1,v2),key notin (v1,v2),key,!key) against an entity's label dictionary at runtime. Handles multi-clause AND semantics and ignores commas inside parentheses.What was explicitly NOT changed
IEntityLabelSelector<TEntity>— untouchedResourceWatcher<TEntity>— untouched; one watcher per entity type regardless of controller countAddController<TImplementation, TEntity>()signature — unchangedTests
LabelSelectorMatcher.Test.cs— 22 unit tests covering all 4 selector operators, multi-clause AND, inner-comma handling, empty/null labelsReconciler.MultiController.Test.cs— 12 tests: catch-all null filter, label match/no-match, ordering, failure short-circuit, entity mutation pass-through,DeletedAsyncpathOperatorBuilder.Test.cs— 3 new tests: multiple registrations in service collection, runtimeGetServicesresolution, no duplicateResourceWatcherregisteredReconciler.Test.csmock: updated fromGetService(IEntityController<T>)toGetService(IEnumerable<IEntityController<T>>)to match the new dispatch pathExample
Backward compatibility
A controller with no
LabelFilteroverride returnsnull, which matches every entity — identical to existing behaviour. No existing code requires changes.