Skip to content

Remove the 'dns.podman' search domain from cluster nodes' resolv.conf#198

Merged
ggiguash merged 2 commits intomicroshift-io:mainfrom
kasturinarra:fix_issue_dns_cncf
Feb 24, 2026
Merged

Remove the 'dns.podman' search domain from cluster nodes' resolv.conf#198
ggiguash merged 2 commits intomicroshift-io:mainfrom
kasturinarra:fix_issue_dns_cncf

Conversation

@kasturinarra
Copy link

@kasturinarra kasturinarra commented Feb 10, 2026

Closes #186

Podman adds 'dns.podman' to /etc/resolv.conf inside containers, creating a 4th search domain beyond the standard 3Kubernetes domains.This triggers a bug in dig 9.9.5 (from jessie-dnsutils:1.7, used by the CNCF e2e DNS conformance tests). The e2e tests run: dig +tcp +noall +answer +search With 4 search domains and ndots:5, dig iterates through all search domains over TCP (all return NXDOMAIN) but fails to fall back to the bare FQDN, producing empty output. The test only writes its result file when dig output is non-empty, so TCP DNS results are never recorded and 3 DNS conformance tests timeout after 10 minutes. Removing 'dns.podman' restores the standard 3-search-domain layout where dig's TCP+search fallback works correctly.

Summary by CodeRabbit

  • Bug Fixes

    • Prevented an undesired DNS search suffix from being appended by the container runtime, improving DNS resolution in affected environments.
  • Chores

    • Updated networking behavior to explicitly provide IP and DNS search settings when not using an isolated network, improving compatibility with DNS conformance tests.

@kasturinarra kasturinarra requested a review from a team as a code owner February 10, 2026 18:20
@kasturinarra kasturinarra marked this pull request as draft February 10, 2026 18:20
@coderabbitai
Copy link

coderabbitai bot commented Feb 10, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9b3ceff and 5022fe7.

📒 Files selected for processing (1)
  • src/cluster_manager.sh

📝 Walkthrough

Walkthrough

Adds a dns search option into the _add_node flow of src/cluster_manager.sh and injects it into the podman run invocation when ISOLATED_NETWORK="0", preventing podman from appending dns.podman to containers' /etc/resolv.conf (workaround for CNCF DNS conformance tests with older dig).

Changes

Cohort / File(s) Summary
Cluster manager script
src/cluster_manager.sh
Introduce dns_opts variable; when ISOLATED_NETWORK="0" add --ip to network options and set dns_opts='--dns-search=.'; pass ${dns_opts} into the podman run command; add comment explaining DNS search-domain workaround for CNCF DNS tests and older dig versions.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Suggested reviewers

  • ggiguash
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly describes the main change: removing 'dns.podman' from cluster nodes' resolv.conf, which is exactly what the PR implements.
Linked Issues check ✅ Passed The PR fully addresses issue #186 by removing the 'dns.podman' search domain to fix CNCF DNS conformance test timeouts.
Out of Scope Changes check ✅ Passed All changes in cluster_manager.sh are directly scoped to removing 'dns.podman' and supporting the DNS conformance fix.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@kasturinarra kasturinarra force-pushed the fix_issue_dns_cncf branch 2 times, most recently from 0fa9e00 to 5a6cbae Compare February 10, 2026 18:43
@kasturinarra kasturinarra force-pushed the fix_issue_dns_cncf branch 23 times, most recently from b877fac to 664a371 Compare February 18, 2026 11:25
@kasturinarra kasturinarra changed the title [WIP]: Fix kindnet POD_SUBNET mismatch in CNCF conformance tests [WIP]: Remove the 'dns.podman' search domain from cluster nodes' resolv.conf Feb 18, 2026
@kasturinarra kasturinarra force-pushed the fix_issue_dns_cncf branch 3 times, most recently from aa67315 to c0181c9 Compare February 18, 2026 16:42
@kasturinarra kasturinarra changed the title [WIP]: Remove the 'dns.podman' search domain from cluster nodes' resolv.conf Closes #186 Remove the 'dns.podman' search domain from cluster nodes' resolv.conf Feb 18, 2026
@kasturinarra kasturinarra marked this pull request as ready for review February 18, 2026 16:49
@kasturinarra
Copy link
Author

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/cncf-conformance.yaml:
- Around line 105-108: The sed command in the microshift node loop can leave a
bare "search" line (invalid) when "search dns.podman" is the only entry; update
the sed expression used in the loop (the command executed inside the podman exec
for nodes microshift-okd-1 and microshift-okd-2) to also delete lines matching a
bare "search" (e.g., adding a `/^search[[:space:]]*$/d` clause) and ensure the
loop uses a per-node echo similar to the firewalld block so each node’s change
is logged; locate the loop that runs podman exec and the sed call and add the
deletion clause and per-node echo there.

@ggiguash ggiguash changed the title Closes #186 Remove the 'dns.podman' search domain from cluster nodes' resolv.conf Remove the 'dns.podman' search domain from cluster nodes' resolv.conf Feb 18, 2026
@kasturinarra kasturinarra force-pushed the fix_issue_dns_cncf branch 2 times, most recently from 4df89bd to 4c3e97a Compare February 19, 2026 09:58
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/cluster_manager.sh`:
- Line 129: The --dns-search option is being added unconditionally and breaks
isolated-network clusters because Podman forbids --dns-search with --network
none; update the node creation logic (where network_name is used, e.g., in
_add_node and callers like cluster_create) to only append the --dns-search
option when network_name != "none" (or when ISOLATED_NETWORK is not set), i.e.,
guard the DNS-related flags behind a conditional that skips adding --dns-search
for the "none" network.

Remove podman's 'dns.podman' search domain to fix DNS conformance
tests. The extra search domain triggers a dig 9.9.5 TCP+search bug
that causes 3 DNS tests to fail.
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/cluster_manager.sh (1)

113-120: Previous critical concern resolved; optional: guard on network_name instead of env var.

The ISOLATED_NETWORK="0" guard correctly keeps dns_opts empty when --network none is in use, fixing the past incompatibility. Works as-is today.

Optionally, tying the guard directly to the network_name parameter (rather than the ambient env var) makes _add_node more self-contained and safe against future callers:

♻️ Proposed refactor — guard on the local parameter
     local network_opts="--network ${network_name}"
     local dns_opts=""
     if [ "${ISOLATED_NETWORK}" = "0" ]; then
         network_opts="${network_opts} --ip ${ip_address}"
+    fi
+    if [ "${network_name}" != "none" ]; then
         # Prevent podman from adding 'dns.podman' to the container's
         # /etc/resolv.conf search domains. The extra search domain
         # breaks CNCF DNS conformance tests with older dig versions.
         dns_opts="--dns-search=."
     fi
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/cluster_manager.sh` around lines 113 - 120, Change the ISOLATED_NETWORK
check to use the local parameter so _add_node is self-contained: instead of
testing ISOLATED_NETWORK, guard on network_name (the function parameter) — e.g.,
only append to network_opts and set dns_opts="--dns-search=." when network_name
is not "none". Update the branch that currently references ISOLATED_NETWORK to
use network_name, keeping the same updates to network_opts and dns_opts; leave
behavior for the isolated/none network unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@src/cluster_manager.sh`:
- Around line 113-120: Change the ISOLATED_NETWORK check to use the local
parameter so _add_node is self-contained: instead of testing ISOLATED_NETWORK,
guard on network_name (the function parameter) — e.g., only append to
network_opts and set dns_opts="--dns-search=." when network_name is not "none".
Update the branch that currently references ISOLATED_NETWORK to use
network_name, keeping the same updates to network_opts and dns_opts; leave
behavior for the isolated/none network unchanged.

@kasturinarra
Copy link
Author

🧹 Nitpick comments (1)

src/cluster_manager.sh (1)> 113-120: Previous critical concern resolved; optional: guard on network_name instead of env var.

The ISOLATED_NETWORK="0" guard correctly keeps dns_opts empty when --network none is in use, fixing the past incompatibility. Works as-is today.
Optionally, tying the guard directly to the network_name parameter (rather than the ambient env var) makes _add_node more self-contained and safe against future callers:

♻️ Proposed refactor — guard on the local parameter

     local network_opts="--network ${network_name}"
     local dns_opts=""
     if [ "${ISOLATED_NETWORK}" = "0" ]; then
         network_opts="${network_opts} --ip ${ip_address}"
+    fi
+    if [ "${network_name}" != "none" ]; then
         # Prevent podman from adding 'dns.podman' to the container's
         # /etc/resolv.conf search domains. The extra search domain
         # breaks CNCF DNS conformance tests with older dig versions.
         dns_opts="--dns-search=."
     fi

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@src/cluster_manager.sh` around lines 113 - 120, Change the ISOLATED_NETWORK
check to use the local parameter so _add_node is self-contained: instead of
testing ISOLATED_NETWORK, guard on network_name (the function parameter) — e.g.,
only append to network_opts and set dns_opts="--dns-search=." when network_name
is not "none". Update the branch that currently references ISOLATED_NETWORK to
use network_name, keeping the same updates to network_opts and dns_opts; leave
behavior for the isolated/none network unchanged.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@src/cluster_manager.sh`:
- Around line 113-120: Change the ISOLATED_NETWORK check to use the local
parameter so _add_node is self-contained: instead of testing ISOLATED_NETWORK,
guard on network_name (the function parameter) — e.g., only append to
network_opts and set dns_opts="--dns-search=." when network_name is not "none".
Update the branch that currently references ISOLATED_NETWORK to use
network_name, keeping the same updates to network_opts and dns_opts; leave
behavior for the isolated/none network unchanged.

Since this works fine now and in the future, let me not modify any of the existing code ?

@ggiguash ggiguash merged commit 3245e2f into microshift-io:main Feb 24, 2026
5 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CNCF dns tests are failing with context deadline exceeded errors

2 participants