Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 25 additions & 104 deletions modules/nodes-cluster-resource-override-move-infra.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,78 +12,32 @@ endif::[]
= Moving the Cluster Resource Override Operator pods

[role="_abstract"]
To reduce the subscription requirements of your cluster, you can move the Cluster Resource Override Operator pods to infrastructure nodes. Move the pods by editing the `Subscription` custom resource (CR) for the Cluster Resource Override Operator and the `ClusterResourceOverride` CR.

By default, the Cluster Resource Override Operator installation process creates an Operator pod and two Cluster Resource Override pods on nodes in the `clusterresourceoverride-operator` namespace. You can move these pods to other nodes, such as infrastructure nodes, as needed.

ifdef::cro[]
You can create and use infrastructure nodes to host only infrastructure components, such as the default router, the integrated container image registry, and the components for cluster metrics and monitoring. These infrastructure nodes are not counted toward the total number of subscriptions that are required to run the environment. For more information about infrastructure nodes, see "Creating infrastructure machine sets".
endif::cro[]

The following examples shows the Cluster Resource Override pods are deployed to control plane nodes and the Cluster Resource Override Operator pod is deployed to a worker node.
.Prerequisites

.Example Cluster Resource Override pods
[source,terminal]
----
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
clusterresourceoverride-786b8c898c-9wrdq 1/1 Running 0 23s 10.128.2.32 ip-10-0-14-183.us-west-2.compute.internal <none> <none>
clusterresourceoverride-786b8c898c-vn2lf 1/1 Running 0 26s 10.130.2.10 ip-10-0-20-140.us-west-2.compute.internal <none> <none>
clusterresourceoverride-operator-6b8b8b656b-lvr62 1/1 Running 0 56m 10.131.0.33 ip-10-0-2-39.us-west-2.compute.internal <none> <none>
----

.Example node list
[source,terminal]
----
NAME STATUS ROLES AGE VERSION
ip-10-0-14-183.us-west-2.compute.internal Ready control-plane,master 65m v1.34.2
ip-10-0-2-39.us-west-2.compute.internal Ready worker 58m v1.34.2
ip-10-0-20-140.us-west-2.compute.internal Ready control-plane,master 65m v1.34.2
ip-10-0-23-244.us-west-2.compute.internal Ready infra 55m v1.34.2
ip-10-0-77-153.us-west-2.compute.internal Ready control-plane,master 65m v1.34.2
ip-10-0-99-108.us-west-2.compute.internal Ready worker 24m v1.34.2
ip-10-0-24-233.us-west-2.compute.internal Ready infra 55m v1.34.2
ip-10-0-88-109.us-west-2.compute.internal Ready worker 24m v1.34.2
ip-10-0-67-453.us-west-2.compute.internal Ready infra 55m v1.34.2
----
* You created one or more infrastructure nodes with a node label such as `node-role.kubernetes.io/infra=""`.

.Procedure

. Move the Cluster Resource Override Operator pod by adding a node selector to the `Subscription` custom resource (CR) for the Cluster Resource Override Operator.
. Move the Cluster Resource Override Operator pod by adding a node selector to its `Subscription` CR:

.. Edit the CR:
.. Edit the CR by running the following command:
+
[source,terminal]
----
$ oc edit -n clusterresourceoverride-operator subscriptions.operators.coreos.com clusterresourceoverride
----

.. Add a node selector to match the node role label on the node where you want to install the Cluster Resource Override Operator pod:
+
[source,terminal]
----
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: clusterresourceoverride
namespace: clusterresourceoverride-operator
# ...
spec:
config:
nodeSelector:
node-role.kubernetes.io/infra: ""
# ...
----
where
+
--
`spec.config.nodeSelector`:: Specifies the role of the node where you want to deploy the Cluster Resource Override Operator pod.
--
.. Add a node selector to match the node role label on the node where you want to move the Cluster Resource Override Operator pod, and add a toleration if the node uses taints:
+
[NOTE]
====
If the infra node uses taints, you need to add a toleration to the `Subscription` CR.

For example:

[source,terminal]
[source,yaml]
----
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
Expand All @@ -99,26 +53,26 @@ spec:
- key: "node-role.kubernetes.io/infra"
operator: "Exists"
effect: "NoSchedule"
# ...
----
+
where:

--
`spec.config.tolerations`:: Specifies a toleration for a taint on the infra node.
--
====
+
`spec.config.nodeSelector`:: Specifies the role label on the node where you want to move the Cluster Resource Override Operator pod
`spec.config.tolerations`:: Specifies the toleration for the node where you want to move the pod. This field is only necessary if the node uses a taint.

. Move the Cluster Resource Override pods by adding a node selector to the `ClusterResourceOverride` custom resource (CR):

.. Edit the CR:
.. Edit the CR by running the following command:
+
[source,terminal]
----
$ oc edit ClusterResourceOverride cluster -n clusterresourceoverride-operator
----

.. Add a node selector to match the node role label on the infra node:
.. Add a node selector to match the node role label on the node, and add a toleration if the node uses taints:
+
[source,terminal]
[source,yaml]
----
apiVersion: operator.autoscaling.openshift.io/v1
kind: ClusterResourceOverride
Expand All @@ -132,65 +86,32 @@ spec:
limitCPUToMemoryPercent: 200
memoryRequestToLimitPercent: 50
deploymentOverrides:
replicas: 1 <1>
replicas: 1
nodeSelector:
node-role.kubernetes.io/infra: "" <2>
# ...
----
where
+
--
`spec.deploymentOverrides.replicas`:: Specifies the number of Cluster Resource Override pods to deploy. The default is `2`. Only one pod is allowed per node. This parameter is optional.
`spec.deploymentOverrides.nodeSelector`:: Specifies the role of the node where you want to deploy the Cluster Resource Override pods. This parameter is optional.
--
+
[NOTE]
====
If the infra node uses taints, you need to add a toleration to the `ClusterResourceOverride` CR.

For example:

[source,terminal]
----
apiVersion: operator.autoscaling.openshift.io/v1
kind: ClusterResourceOverride
metadata:
name: cluster
# ...
spec:
podResourceOverride:
spec:
memoryRequestToLimitPercent: 50
cpuRequestToLimitPercent: 25
limitCPUToMemoryPercent: 200
deploymentOverrides:
replicas: 3
nodeSelector:
node-role.kubernetes.io/worker: ""
tolerations: <1>
node-role.kubernetes.io/infra: ""
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I am sure this is correct, but this should get a QA ✔️

tolerations:
- key: "key"
operator: "Equal"
value: "value"
effect: "NoSchedule"
# ...
----
+
where:
+
--
`spec.config.tolerations`:: Specifies a toleration for a taint on the infra node.
--
====
`spec.deploymentOverrides.replicas`:: Optional: specifies the number of Cluster Resource Override pods to deploy. The default is `2`. Only one pod is allowed per node.
`spec.deploymentOverrides.nodeSelector`:: Specifies the role label on the node where you want to move the Cluster Resource Override Operator pod
`spec.deploymentOverrides.tolerations`:: Specifies the toleration for the node where you want to move the pod. This field is only necessary if the node uses a taint.

.Verification

* You can verify that the pods have moved by using the following command:
* You can verify that the pods have moved to the nodes with the label you specified by using the following command:
+
[source,terminal]
----
$ oc get pods -n clusterresourceoverride-operator -o wide
----
+
The Cluster Resource Override pods are now deployed to the infra nodes.
+
.Example output
[source,terminal]
----
Expand Down
Loading