diff --git a/modules/nodes-cluster-resource-override-move-infra.adoc b/modules/nodes-cluster-resource-override-move-infra.adoc index fcf66ff7e62b..f3c98b1ede57 100644 --- a/modules/nodes-cluster-resource-override-move-infra.adoc +++ b/modules/nodes-cluster-resource-override-move-infra.adoc @@ -12,78 +12,32 @@ endif::[] = Moving the Cluster Resource Override Operator pods [role="_abstract"] +To reduce the subscription requirements of your cluster, you can move the Cluster Resource Override Operator pods to infrastructure nodes. Move the pods by editing the `Subscription` custom resource (CR) for the Cluster Resource Override Operator and the `ClusterResourceOverride` CR. + By default, the Cluster Resource Override Operator installation process creates an Operator pod and two Cluster Resource Override pods on nodes in the `clusterresourceoverride-operator` namespace. You can move these pods to other nodes, such as infrastructure nodes, as needed. ifdef::cro[] You can create and use infrastructure nodes to host only infrastructure components, such as the default router, the integrated container image registry, and the components for cluster metrics and monitoring. These infrastructure nodes are not counted toward the total number of subscriptions that are required to run the environment. For more information about infrastructure nodes, see "Creating infrastructure machine sets". endif::cro[] -The following examples shows the Cluster Resource Override pods are deployed to control plane nodes and the Cluster Resource Override Operator pod is deployed to a worker node. +.Prerequisites -.Example Cluster Resource Override pods -[source,terminal] ----- -NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES -clusterresourceoverride-786b8c898c-9wrdq 1/1 Running 0 23s 10.128.2.32 ip-10-0-14-183.us-west-2.compute.internal -clusterresourceoverride-786b8c898c-vn2lf 1/1 Running 0 26s 10.130.2.10 ip-10-0-20-140.us-west-2.compute.internal -clusterresourceoverride-operator-6b8b8b656b-lvr62 1/1 Running 0 56m 10.131.0.33 ip-10-0-2-39.us-west-2.compute.internal ----- - -.Example node list -[source,terminal] ----- -NAME STATUS ROLES AGE VERSION -ip-10-0-14-183.us-west-2.compute.internal Ready control-plane,master 65m v1.34.2 -ip-10-0-2-39.us-west-2.compute.internal Ready worker 58m v1.34.2 -ip-10-0-20-140.us-west-2.compute.internal Ready control-plane,master 65m v1.34.2 -ip-10-0-23-244.us-west-2.compute.internal Ready infra 55m v1.34.2 -ip-10-0-77-153.us-west-2.compute.internal Ready control-plane,master 65m v1.34.2 -ip-10-0-99-108.us-west-2.compute.internal Ready worker 24m v1.34.2 -ip-10-0-24-233.us-west-2.compute.internal Ready infra 55m v1.34.2 -ip-10-0-88-109.us-west-2.compute.internal Ready worker 24m v1.34.2 -ip-10-0-67-453.us-west-2.compute.internal Ready infra 55m v1.34.2 ----- +* You created one or more infrastructure nodes with a node label such as `node-role.kubernetes.io/infra=""`. .Procedure -. Move the Cluster Resource Override Operator pod by adding a node selector to the `Subscription` custom resource (CR) for the Cluster Resource Override Operator. +. Move the Cluster Resource Override Operator pod by adding a node selector to its `Subscription` CR: -.. Edit the CR: +.. Edit the CR by running the following command: + [source,terminal] ---- $ oc edit -n clusterresourceoverride-operator subscriptions.operators.coreos.com clusterresourceoverride ---- -.. Add a node selector to match the node role label on the node where you want to install the Cluster Resource Override Operator pod: -+ -[source,terminal] ----- -apiVersion: operators.coreos.com/v1alpha1 -kind: Subscription -metadata: - name: clusterresourceoverride - namespace: clusterresourceoverride-operator -# ... -spec: - config: - nodeSelector: - node-role.kubernetes.io/infra: "" -# ... ----- -where -+ --- -`spec.config.nodeSelector`:: Specifies the role of the node where you want to deploy the Cluster Resource Override Operator pod. --- +.. Add a node selector to match the node role label on the node where you want to move the Cluster Resource Override Operator pod, and add a toleration if the node uses taints: + -[NOTE] -==== -If the infra node uses taints, you need to add a toleration to the `Subscription` CR. - -For example: - -[source,terminal] +[source,yaml] ---- apiVersion: operators.coreos.com/v1alpha1 kind: Subscription @@ -99,26 +53,26 @@ spec: - key: "node-role.kubernetes.io/infra" operator: "Exists" effect: "NoSchedule" +# ... ---- ++ where: - --- -`spec.config.tolerations`:: Specifies a toleration for a taint on the infra node. --- -==== ++ +`spec.config.nodeSelector`:: Specifies the role label on the node where you want to move the Cluster Resource Override Operator pod +`spec.config.tolerations`:: Specifies the toleration for the node where you want to move the pod. This field is only necessary if the node uses a taint. . Move the Cluster Resource Override pods by adding a node selector to the `ClusterResourceOverride` custom resource (CR): -.. Edit the CR: +.. Edit the CR by running the following command: + [source,terminal] ---- $ oc edit ClusterResourceOverride cluster -n clusterresourceoverride-operator ---- -.. Add a node selector to match the node role label on the infra node: +.. Add a node selector to match the node role label on the node, and add a toleration if the node uses taints: + -[source,terminal] +[source,yaml] ---- apiVersion: operator.autoscaling.openshift.io/v1 kind: ClusterResourceOverride @@ -132,65 +86,32 @@ spec: limitCPUToMemoryPercent: 200 memoryRequestToLimitPercent: 50 deploymentOverrides: - replicas: 1 <1> + replicas: 1 nodeSelector: - node-role.kubernetes.io/infra: "" <2> -# ... ----- -where -+ --- -`spec.deploymentOverrides.replicas`:: Specifies the number of Cluster Resource Override pods to deploy. The default is `2`. Only one pod is allowed per node. This parameter is optional. -`spec.deploymentOverrides.nodeSelector`:: Specifies the role of the node where you want to deploy the Cluster Resource Override pods. This parameter is optional. --- -+ -[NOTE] -==== -If the infra node uses taints, you need to add a toleration to the `ClusterResourceOverride` CR. - -For example: - -[source,terminal] ----- -apiVersion: operator.autoscaling.openshift.io/v1 -kind: ClusterResourceOverride -metadata: - name: cluster -# ... -spec: - podResourceOverride: - spec: - memoryRequestToLimitPercent: 50 - cpuRequestToLimitPercent: 25 - limitCPUToMemoryPercent: 200 - deploymentOverrides: - replicas: 3 - nodeSelector: - node-role.kubernetes.io/worker: "" - tolerations: <1> + node-role.kubernetes.io/infra: "" + tolerations: - key: "key" operator: "Equal" value: "value" effect: "NoSchedule" +# ... ---- ++ where: + --- -`spec.config.tolerations`:: Specifies a toleration for a taint on the infra node. --- -==== +`spec.deploymentOverrides.replicas`:: Optional: specifies the number of Cluster Resource Override pods to deploy. The default is `2`. Only one pod is allowed per node. +`spec.deploymentOverrides.nodeSelector`:: Specifies the role label on the node where you want to move the Cluster Resource Override Operator pod +`spec.deploymentOverrides.tolerations`:: Specifies the toleration for the node where you want to move the pod. This field is only necessary if the node uses a taint. .Verification -* You can verify that the pods have moved by using the following command: +* You can verify that the pods have moved to the nodes with the label you specified by using the following command: + [source,terminal] ---- $ oc get pods -n clusterresourceoverride-operator -o wide ---- + -The Cluster Resource Override pods are now deployed to the infra nodes. -+ .Example output [source,terminal] ---- diff --git a/modules/nodes-pods-vertical-autoscaler-moving-vpa.adoc b/modules/nodes-pods-vertical-autoscaler-moving-vpa.adoc index ffe7cb2c9748..c4fbe1916be4 100644 --- a/modules/nodes-pods-vertical-autoscaler-moving-vpa.adoc +++ b/modules/nodes-pods-vertical-autoscaler-moving-vpa.adoc @@ -3,77 +3,37 @@ // * machine_management/creating-infrastructure-machinesets.adoc // * nodes/pods/nodes-pods-vertical-autoscaler -ifeval::["{context}" == "nodes-pods-vertical-autoscaler"] -:vpa: -endif::[] -ifeval::["{context}" == "creating-infrastructure-machinesets"] -:machinemgmt: -endif::[] - :_mod-docs-content-type: PROCEDURE [id="infrastructure-moving-vpa_{context}"] = Moving the Vertical Pod Autoscaler Operator components -ifdef::machinemgmt[] -The Vertical Pod Autoscaler Operator (VPA) consists of three components: the recommender, updater, and admission controller. The Operator and each component has its own pod in the VPA namespace on the control plane nodes. You can move the VPA Operator and component pods to infrastructure nodes by adding a node selector to the VPA subscription and the `VerticalPodAutoscalerController` CR. -endif::machinemgmt[] -ifdef::vpa[] -The Vertical Pod Autoscaler Operator (VPA) and each component has its own pod in the VPA namespace on the control plane nodes. You can move the VPA Operator and component pods to infrastructure or worker nodes by adding a node selector to the VPA subscription and the `VerticalPodAutoscalerController` CR. +[role="_abstract"] +To reduce the subscription requirements of your cluster, you can move the Vertical Pod Autoscaler Operator (VPA) pods to infrastructure nodes. Move the pods by editing the `Subscription` custom resource (CR) for the VPA and the `VerticalPodAutoscaler` CR. + +The Vertical Pod Autoscaler Operator (VPA) consists of three components: the recommender, updater, and admission controller. The Operator and each component has its own pod in the VPA namespace on the control plane nodes. You can create and use infrastructure nodes to host only infrastructure components. For example, the default router, the integrated container image registry, and the components for cluster metrics and monitoring. These infrastructure nodes are not counted toward the total number of subscriptions that are required to run the environment. For more information, see _Creating infrastructure machine sets_. You can move the components to the same node or separate nodes as appropriate for your organization. -endif::vpa[] -The following example shows the default deployment of the VPA pods to the control plane nodes. +.Prerequisites -.Example output -[source,terminal] ----- -NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES -vertical-pod-autoscaler-operator-6c75fcc9cd-5pb6z 1/1 Running 0 7m59s 10.128.2.24 c416-tfsbj-master-1 -vpa-admission-plugin-default-6cb78d6f8b-rpcrj 1/1 Running 0 5m37s 10.129.2.22 c416-tfsbj-master-1 -vpa-recommender-default-66846bd94c-dsmpp 1/1 Running 0 5m37s 10.129.2.20 c416-tfsbj-master-0 -vpa-updater-default-db8b58df-2nkvf 1/1 Running 0 5m37s 10.129.2.21 c416-tfsbj-master-1 ----- +* You created one or more infrastructure nodes with a node label such as `node-role.kubernetes.io/infra=""`. .Procedure -ifdef::machinemgmt[] -. Move the VPA Operator pod by adding a node selector to the `Subscription` custom resource (CR) for the VPA Operator: +. Move the VPA Operator pod by adding a node selector to the `Subscription` CR for the VPA Operator: -.. Edit the CR: +.. Edit the CR by running the following command: + [source,terminal] ---- $ oc edit Subscription vertical-pod-autoscaler -n openshift-vertical-pod-autoscaler ---- -.. Add a node selector to match the node role label on the infra node: -+ -[source,terminal] ----- -apiVersion: operators.coreos.com/v1alpha1 -kind: Subscription -metadata: - labels: - operators.coreos.com/vertical-pod-autoscaler.openshift-vertical-pod-autoscaler: "" - name: vertical-pod-autoscaler -# ... -spec: - config: - nodeSelector: - node-role.kubernetes.io/infra: "" <1> ----- -<1> Specifies the node role of an infra node. +.. Add a node selector to match the node role label on the node where you want to install the VPA Operator pod: + -[NOTE] -==== -If the infra node uses taints, you need to add a toleration to the `Subscription` CR. - -For example: - -[source,terminal] +[source,yaml] ---- apiVersion: operators.coreos.com/v1alpha1 kind: Subscription @@ -85,164 +45,21 @@ metadata: spec: config: nodeSelector: - node-role.kubernetes.io/infra: "" - tolerations: <1> - - key: "node-role.kubernetes.io/infra" + node-role.kubernetes.io/: "" + tolerations: + - key: "node-role.kubernetes.io/" operator: "Exists" effect: "NoSchedule" ---- -==== -<1> Specifies a toleration for a taint on the infra node. - -. Move each VPA component by adding node selectors to the `VerticalPodAutoscaler` custom resource (CR): - -.. Edit the CR: -+ -[source,terminal] ----- -$ oc edit VerticalPodAutoscalerController default -n openshift-vertical-pod-autoscaler ----- - -.. Add node selectors to match the node role label on the infra node: -+ -[source,terminal] ----- -apiVersion: autoscaling.openshift.io/v1 -kind: VerticalPodAutoscalerController -metadata: - name: default - namespace: openshift-vertical-pod-autoscaler -# ... -spec: - deploymentOverrides: - admission: - container: - resources: {} - nodeSelector: - node-role.kubernetes.io/infra: "" <1> - recommender: - container: - resources: {} - nodeSelector: - node-role.kubernetes.io/infra: "" <2> - updater: - container: - resources: {} - nodeSelector: - node-role.kubernetes.io/infra: "" <3> ----- -<1> Optional: Specifies the node role for the VPA admission pod. -<2> Optional: Specifies the node role for the VPA recommender pod. -<3> Optional: Specifies the node role for the VPA updater pod. -+ -[NOTE] -==== -If a target node uses taints, you need to add a toleration to the `VerticalPodAutoscalerController` CR. - -For example: - -[source,terminal] ----- -apiVersion: autoscaling.openshift.io/v1 -kind: VerticalPodAutoscalerController -metadata: - name: default - namespace: openshift-vertical-pod-autoscaler -# ... -spec: - deploymentOverrides: - admission: - container: - resources: {} - nodeSelector: - node-role.kubernetes.io/infra: "" - tolerations: <1> - - key: "my-example-node-taint-key" - operator: "Exists" - effect: "NoSchedule" - recommender: - container: - resources: {} - nodeSelector: - node-role.kubernetes.io/infra: "" - tolerations: <2> - - key: "my-example-node-taint-key" - operator: "Exists" - effect: "NoSchedule" - updater: - container: - resources: {} - nodeSelector: - node-role.kubernetes.io/infra: "" - tolerations: <3> - - key: "my-example-node-taint-key" - operator: "Exists" - effect: "NoSchedule" ----- -==== -<1> Specifies a toleration for the admission controller pod for a taint on the infra node. -<2> Specifies a toleration for the recommender pod for a taint on the infra node. -<3> Specifies a toleration for the updater pod for a taint on the infra node. -endif::machinemgmt[] - -ifdef::vpa[] -. Move the VPA Operator pod by adding a node selector to the `Subscription` custom resource (CR) for the VPA Operator: - -.. Edit the CR: -+ -[source,terminal] ----- -$ oc edit Subscription vertical-pod-autoscaler -n openshift-vertical-pod-autoscaler ----- - -.. Add a node selector to match the node role label on the node where you want to install the VPA Operator pod: + -[source,terminal] ----- -apiVersion: operators.coreos.com/v1alpha1 -kind: Subscription -metadata: - labels: - operators.coreos.com/vertical-pod-autoscaler.openshift-vertical-pod-autoscaler: "" - name: vertical-pod-autoscaler -# ... -spec: - config: - nodeSelector: - node-role.kubernetes.io/: "" <1> ----- -<1> Specifies the node role of the node where you want to move the VPA Operator pod. +where: + -[NOTE] -==== -If the infra node uses taints, you need to add a toleration to the `Subscription` CR. - -For example: - -[source,terminal] ----- -apiVersion: operators.coreos.com/v1alpha1 -kind: Subscription -metadata: - labels: - operators.coreos.com/vertical-pod-autoscaler.openshift-vertical-pod-autoscaler: "" - name: vertical-pod-autoscaler -# ... -spec: - config: - nodeSelector: - node-role.kubernetes.io/infra: "" - tolerations: <1> - - key: "node-role.kubernetes.io/infra" - operator: "Exists" - effect: "NoSchedule" ----- -==== -<1> Specifies a toleration for a taint on the node where you want to move the VPA Operator pod. +`spec.config.nodeSelector`:: Specifies the node selector of the node where you want to move the VPA Operator pod and `` is the node role. For example, to move the pod to an infrastructure node, this value would be `node-role.kubernetes.io/infra: ""`. +`spec.config.tolerations`:: Specifies a toleration for a taint on the node where you want to move the VPA Operator pod. This parameter is only necessary if the destination node uses a taint. . Move each VPA component by adding node selectors to the `VerticalPodAutoscaler` custom resource (CR): -.. Edit the CR: +.. Edit the CR by running the following command: + [source,terminal] ---- @@ -251,7 +68,7 @@ $ oc edit VerticalPodAutoscalerController default -n openshift-vertical-pod-auto .. Add node selectors to match the node role label on the node where you want to install the VPA components: + -[source,terminal] +[source,yaml] ---- apiVersion: autoscaling.openshift.io/v1 kind: VerticalPodAutoscalerController @@ -265,71 +82,39 @@ spec: container: resources: {} nodeSelector: - node-role.kubernetes.io/: "" <1> - recommender: - container: - resources: {} - nodeSelector: - node-role.kubernetes.io/: "" <2> - updater: - container: - resources: {} - nodeSelector: - node-role.kubernetes.io/: "" <3> ----- -<1> Optional: Specifies the node role for the VPA admission pod. -<2> Optional: Specifies the node role for the VPA recommender pod. -<3> Optional: Specifies the node role for the VPA updater pod. -+ -[NOTE] -==== -If a target node uses taints, you need to add a toleration to the `VerticalPodAutoscalerController` CR. - -For example: - -[source,terminal] ----- -apiVersion: autoscaling.openshift.io/v1 -kind: VerticalPodAutoscalerController -metadata: - name: default - namespace: openshift-vertical-pod-autoscaler -# ... -spec: - deploymentOverrides: - admission: - container: - resources: {} - nodeSelector: - node-role.kubernetes.io/worker: "" - tolerations: <1> - - key: "my-example-node-taint-key" + node-role.kubernetes.io/: "" + tolerations: + - key: "node-role.kubernetes.io/" operator: "Exists" effect: "NoSchedule" recommender: container: resources: {} nodeSelector: - node-role.kubernetes.io/worker: "" - tolerations: <2> - - key: "my-example-node-taint-key" + node-role.kubernetes.io/: "" + tolerations: + - key: "node-role.kubernetes.io/" operator: "Exists" effect: "NoSchedule" updater: container: resources: {} nodeSelector: - node-role.kubernetes.io/worker: "" - tolerations: <3> - - key: "my-example-node-taint-key" + node-role.kubernetes.io/: "" + tolerations: + - key: "node-role.kubernetes.io/" operator: "Exists" effect: "NoSchedule" ---- -==== -<1> Specifies a toleration for the admission controller pod for a taint on the node where you want to install the pod. -<2> Specifies a toleration for the recommender pod for a taint on the node where you want to install the pod. -<3> Specifies a toleration for the updater pod for a taint on the node where you want to install the pod. -endif::vpa[] ++ +where: ++ +`spec.deploymentOverrides.admission.nodeSelector`:: Specifies the node selector of the node where you want to move the admission pod and `` is the node role. For example, to move the pod to an infrastructure node, this value would be `node-role.kubernetes.io/infra: ""`. +`spec.deploymentOverrides.admission.tolerations`:: Specifies a toleration for a taint on the node where you want to move the admission pod and `` is the node role. For example, to tolerate a taint on an infrastructure node, the toleration key value would be `node-role.kubernetes.io/infra: ""`. This parameter is only necessary if the destination node uses a taint. +`spec.deploymentOverrides.recommender.nodeSelector`:: Specifies the node selector of the node where you want to move the recommender pod and `` is the node role. For example, to move the pod to an infrastructure node, this value would be `node-role.kubernetes.io/infra: ""`. +`spec.deploymentOverrides.recommender.tolerations`:: Specifies a toleration for a taint on the node where you want to move the recommender pod and `` is the node role. For example, to tolerate a taint on an infrastructure node, the toleration key value would be `node-role.kubernetes.io/infra: ""`. This parameter is only necessary if the destination node uses a taint. +`spec.deploymentOverrides.updater.nodeSelector`:: Specifies the node selector of the node where you want to move the updater pod and `` is the node role. For example, to move the pod to an infrastructure node, this value would be `node-role.kubernetes.io/infra: ""`. +`spec.deploymentOverrides.updater.tolerations`:: Specifies a toleration for a taint on the node where you want to move the updater pod and `` is the node role. For example, to tolerate a taint on an infrastructure node, the toleration key value would be `node-role.kubernetes.io/infra: ""`. This parameter is only necessary if the destination node uses a taint. .Verification @@ -351,10 +136,3 @@ vpa-admission-plugin-default-6cb78d6f8b-rpcrj 1/1 Running 0 vpa-recommender-default-66846bd94c-dsmpp 1/1 Running 0 5m37s 10.129.2.20 c416-tfsbj-infra-eastus1-lrgj8 vpa-updater-default-db8b58df-2nkvf 1/1 Running 0 5m37s 10.129.2.21 c416-tfsbj-infra-eastus1-lrgj8 ---- - -ifeval::["{context}" == "nodes-pods-vertical-autoscaler"] -:!vpa: -endif::[] -ifeval::["{context}" == "creating-infrastructure-machinesets"] -:!machinemgmt: -endif::[]