OCPBUGS-90507: Update CNV swap tests for mandatory drop-in directory

[BhargaviGudi]

Problem

TC6 test fails 100% of the time with 15-minute timeout, blocking daily CI runs.

Root Cause

MCO PR #6044 made /etc/openshift/kubelet.conf.d/ mandatory on ALL nodes (masters, workers) and now contains:

• 20-auto-sizing.conf (required for kubelet startup)
• 99-kubelet-limited-swap.conf (optional CNV swap config)

TC6 Invalid: Deletes entire directory to test auto-recreation, which removes 20-auto-sizing.conf and breaks kubelet (15-min timeout, node degradation).

TC5 Invalid: Assumes control plane ignores drop-in configs. Reality: control plane DOES apply LimitedSwap from drop-in (test fails) - Per MCO PR #6044, the drop-in directory is configured for control plane nodes

TC1 Outdated: Expected directory only on worker nodes. Now directory is mandatory on ALL nodes per MCO #6044.

Solution

• Removed TC6 - Invalid test case
• Removed TC5 - Invalid assumption (control plane applies drop-in configs)
• Fixed TC1 - Updated to verify directory exists on control plane nodes (not just workers)
• Renumbered - TC7-TC14 → TC5-TC12 after removals

Summary by CodeRabbit

• Tests
  • Reorganized and expanded LimitedSwap validation test suite with enhanced coverage including configuration permissions, cluster stability, and multi-node consistency
  • Added tests for swap sizing scenarios, configuration precedence handling, and observability metrics
  • Improved validation for impact on non-CNV nodes and various swap configurations

[openshift-merge-bot]

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[openshift-ci-robot]

@BhargaviGudi: This pull request references Jira Issue OCPBUGS-90507, which is invalid:

• expected the bug to target only the "5.0.0" version, but multiple target versions were set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Problem

TC6 test fails 100% of the time with 15-minute timeout, blocking daily CI runs.

Root Cause

TC6 deletes /etc/openshift/kubelet.conf.d/ directory to test auto-recreation.

On 5.0, this directory now contains:

• 20-auto-sizing.conf (required for kubelet)
• 99-kubelet-limited-swap.conf (CNV swap config)

When TC6 deletes the entire directory:

1. Removes 20-auto-sizing.conf (required for kubelet startup)
2. Kubelet fails to start
3. oc debug node times out after 15 minutes
4. Node becomes degraded

Solution

Remove TC6 - it's now an invalid test case on 5.0 due to shared directory usage.

Renumbered TC7-TC14 to TC6-TC13 after removal.

Testing

• ✅ make update passed
• ✅ make build passed
• ✅ make verify passed

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 4d16639c-8190-4f51-8ad4-6d236c867fb0

📥 Commits

Reviewing files that changed from the base of the PR and between 80e03ef and e8f3813.

📒 Files selected for processing (1)

• test/extended/node/node_swap_cnv.go

🚧 Files skipped from review as they are similar to previous changes (1)

• test/extended/node/node_swap_cnv.go

---

Walkthrough

The CNV kubelet LimitedSwap drop-in test suite in node_swap_cnv.go updates TC1's control-plane validation to expect the drop-in directory to exist on all nodes (previously absent on control planes), then reorganizes test cases TC5 through TC12 with new behavioral scopes: security/permissions validation, cluster stability monitoring, non-CNV worker isolation, multi-file precedence, multi-node consistency, OS swap disabled behavior, various swap sizes, and Prometheus metrics observability.

Changes

CNV LimitedSwap validation test suite reorganization

Layer / File(s)	Summary
TC1 control-plane drop-in directory expectation test/extended/node/node_swap_cnv.go	TC1 header and control-plane/master-node validation branch are updated to assert the drop-in directory exists on all nodes, including control planes with root:root ownership verification, reversing the prior expectation that it should not exist on control planes.
TC5–TC7 security, stability, and isolation validation test/extended/node/node_swap_cnv.go	TC5 is replaced with drop-in directory and file permissions/ownership checks; TC6 is redefined as cluster stability and performance monitoring after enabling LimitedSwap, checking readiness and memory-pressure absence; TC7 is reintroduced to verify swap configuration resets when CNV labels are removed from worker nodes.
TC8–TC10 multi-file precedence, consistency, and OS-level behavior test/extended/node/node_swap_cnv.go	TC8 validates file precedence when multiple conflicting drop-in files exist; TC9 implements multi-node checksum consistency validation across CNV nodes and verifies no configuration drift after restart; TC10 is renumbered as the OS swap disabled scenario, confirming kubelet readiness, LimitedSwap behavior loading, and absence of fatal swap errors when OS-level swap is disabled.
TC11–TC12 swap size variations and metrics observability test/extended/node/node_swap_cnv.go	TC11 validates kubelet behavior across multiple swap sizes by creating temporary swap files, restarting kubelet, verifying readiness and LimitedSwap behavior, and checking /proc/meminfo swap totals; TC12 validates swap metrics by conditionally querying Prometheus node swap metrics and kubelet endpoints, verifying swap-related fields are present and nodes remain Ready.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Test Structure And Quality	⚠️ Warning	Tests violate single responsibility (15-30 By blocks per test mixing multiple concerns) and 53% of assertions lack meaningful failure messages required for diagnostics.	Split multi-purpose tests into focused units; add failure messages to all assertions per pattern: Expect(x).To(y, "descriptive message for failure")

✅ Passed checks (14 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: updating CNV swap tests to address the mandatory drop-in directory requirement from MCO PR #6044.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	All Ginkgo test titles (It, Describe) are static and deterministic. No dynamic content like node names, pod names, timestamps, UUIDs, or IP addresses found in test declarations. TC1-TC12 use stable...
Microshift Test Compatibility	✅ Passed	All tests in this file are protected by a BeforeAll hook that checks for MicroShift and skips the entire test suite. No individual tests will attempt to use unavailable APIs on MicroShift.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	All tests that require multi-node clusters are protected: TC2-TC6, TC8-TC12 have skipOnSingleNodeTopology guards. TC1 filters pure worker nodes (empty in SNO) but checks control plane nodes as fall...
Topology-Aware Scheduling Compatibility	✅ Passed	PR modifies test code only (test/extended/node/node_swap_cnv.go), not deployment manifests, operators, or controllers. No scheduling constraints introduced.
Ote Binary Stdout Contract	✅ Passed	File is a Ginkgo v2 test with no process-level stdout writes. All logging uses framework.Logf (intercepted by framework), top-level vars use safe path construction, no main/init/TestMain functions...
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	Test contains no IPv4 hardcoded addresses, IPv6-incompatible patterns, or required external internet connectivity. TC12's Prometheus queries are optional with graceful fallback, suitable for IPv6-o...
No-Weak-Crypto	✅ Passed	The PR modifies only a test file using md5sum CLI tool for configuration consistency checks across nodes, not for security/cryptographic purposes. No weak crypto libraries, DES/RC4/3DES/Blowfish, E...
Container-Privileges	✅ Passed	No privileged container specifications, hostPID/Network/IPC, SYS_ADMIN capabilities, or allowPrivilegeEscalation: true found in any manifests or configurations in the PR.
No-Sensitive-Data-In-Logs	✅ Passed	The pull request contains 319 logging statements examining kubelet swap configuration. Comprehensive review shows no passwords, tokens, API keys, PII, session IDs, or sensitive credentials are logg...

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: BhargaviGudi
Once this PR has been reviewed and has the lgtm label, please assign rphillips for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• test/extended/node/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[BhargaviGudi]

/jira refresh

[openshift-ci-robot]

@BhargaviGudi: This pull request references Jira Issue OCPBUGS-90507, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.0.0) matches configured target version for branch (5.0.0)
• bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[BhargaviGudi]

/payload-job periodic-ci-openshift-release-main-nightly-5.0-e2e-aws-disruptive-longrunning

[openshift-ci]

@BhargaviGudi: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-release-main-nightly-5.0-e2e-aws-disruptive-longrunning

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/01f016b0-6bcf-11f1-9a52-cb1165f84022-0

[openshift-ci-robot]

@BhargaviGudi: This pull request references Jira Issue OCPBUGS-90507, which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.0.0) matches configured target version for branch (5.0.0)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Details

In response to this:

Problem

TC6 test fails 100% of the time with 15-minute timeout, blocking daily CI runs.

Root Cause

TC6 deletes /etc/openshift/kubelet.conf.d/ directory to test auto-recreation.

On 5.0, this directory now contains:

• 20-auto-sizing.conf (required for kubelet)
• 99-kubelet-limited-swap.conf (CNV swap config)

When TC6 deletes the entire directory:

1. Removes 20-auto-sizing.conf (required for kubelet startup)
2. Kubelet fails to start
3. oc debug node times out after 15 minutes
4. Node becomes degraded

Solution

Remove TC6 - it's now an invalid test case on 5.0 due to shared directory usage.

Renumbered TC7-TC14 to TC6-TC13 after removal.

Testing

• ✅ make update passed
• ✅ make build passed
• ✅ make verify passed

Summary by CodeRabbit

• Tests
• Reorganized and renumbered the LimitedSwap validation suite to improve coverage across control-plane and worker nodes.
• Added/updated scenarios for CNV drop-in presence, permissions validation, cluster stability after enabling LimitedSwap, and ensuring non-CNV workers remain unaffected.
• Extended coverage for conflicting drop-in precedence, multi-node consistency checks, OS swap-disabled behavior, swap-size permutations, and verification of observability/Prometheus swap metrics.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/extended/node/node_swap_cnv.go`:
- Around line 163-176: The test can pass even when getNodesByLabel returns zero
control plane nodes, because the for loop iterating over controlPlaneNodes will
never execute but the test still reports success at line 176, creating a false
positive. Add an assertion after the getNodesByLabel call (after line 164) using
the o.Expect pattern to verify that len(controlPlaneNodes) is greater than zero,
ensuring the test actually validates control plane nodes before claiming TC1
passed.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 14a1295c-8216-4db3-b7cc-2a87656af3b9

📥 Commits

Reviewing files that changed from the base of the PR and between 6ebe642 and 80e03ef.

📒 Files selected for processing (1)

• test/extended/node/node_swap_cnv.go

[openshift-ci-robot]

@BhargaviGudi: This pull request references Jira Issue OCPBUGS-90507, which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.0.0) matches configured target version for branch (5.0.0)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Problem

TC6 test fails 100% of the time with 15-minute timeout, blocking daily CI runs.

Root Cause

MCO PR #6044 made /etc/openshift/kubelet.conf.d/ mandatory on ALL nodes (masters, workers) and now contains:

• 20-auto-sizing.conf (required for kubelet startup)
• 99-kubelet-limited-swap.conf (optional CNV swap config)

TC6 Invalid: Deletes entire directory to test auto-recreation, which removes 20-auto-sizing.conf and breaks kubelet (15-min timeout, node degradation).

TC5 Invalid: Assumes control plane ignores drop-in configs. Reality: control plane DOES apply LimitedSwap from drop-in (test fails) - Per MCO PR #6044, the drop-in directory is configured for control plane nodes

TC1 Outdated: Expected directory only on worker nodes. Now directory is mandatory on ALL nodes per MCO #6044.

Solution

• Removed TC6 - Invalid test case
• Removed TC5 - Invalid assumption (control plane applies drop-in configs)
• Fixed TC1 - Updated to verify directory exists on control plane nodes (not just workers)
• Renumbered - TC7-TC14 → TC5-TC12 after removals

Summary by CodeRabbit

• Tests
• Reorganized and expanded LimitedSwap validation test suite with enhanced coverage including configuration permissions, cluster stability, and multi-node consistency
• Added tests for swap sizing scenarios, configuration precedence handling, and observability metrics
• Improved validation for impact on non-CNV nodes and various swap configurations

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[BhargaviGudi]

/payload-job periodic-ci-openshift-release-main-nightly-5.0-e2e-aws-disruptive-longrunning

[openshift-ci]

@BhargaviGudi: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-release-main-nightly-5.0-e2e-aws-disruptive-longrunning

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/997954f0-6e0a-11f1-8723-a88d645fe1bf-0

[openshift-merge-bot]

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi