Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion modules/cnf-configuring-high-priority-workload-pods.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,15 @@
[id="cnf-configuring-high-priority-workload-pods_{context}"]
= Disabling power saving mode for high priority pods

[role="_abstract"]
You can configure pods to ensure that high priority workloads are unaffected when you configure power saving for the node that the workloads run on.

When you configure a node with a power saving configuration, you must configure high priority workloads with performance configuration at the pod level, which means that the configuration applies to all the cores used by the pod.

By disabling P-states and C-states at the pod level, you can configure high priority workloads for best performance and lowest latency.

.Configuration for high priority workloads
The following table describes the configuration for high priority workloads:

[cols="1,2,3", options="header"]

|===
Expand Down
2 changes: 2 additions & 0 deletions modules/cnf-disabling-cpu-cfs-quota.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
//
// * scalability_and_performance/low_latency_tuning/cnf-provisioning-low-latency-workloads.adoc

:_mod-docs-content-type: CONCEPT
[id="cnf-disabling-cpu-cfs-quota_{context}"]
= Disabling CPU CFS quota

[role="_abstract"]
To eliminate CPU throttling for pinned pods, create a pod with the `cpu-quota.crio.io: "disable"` annotation. This annotation disables the CPU completely fair scheduler (CFS) quota when the pod runs.

.Example pod specification with `cpu-quota.crio.io` disabled
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
[id="cnf-disabling-interrupt-processing-for-individual-pods_{context}"]
= Configuring interrupt processing for individual pods

[role="_abstract"]
To achieve low latency for workloads, some containers require that the CPUs they are pinned to do not process device interrupts. You can use the `irq-load-balancing.crio.io` pod annotation to control whether device interrupts are processed on CPUs where the pinned containers are running.

The annotation supports the following values:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@
[id="cnf-node-tuning-operator-creating-pod-with-guaranteed-qos-class_{context}"]
= Creating a pod with a guaranteed QoS class

You can create a pod with a quality of service (QoS) class of `Guaranteed` for high-performance workloads. Configuring a pod with a QoS class of `Guaranteed` ensures that the pod has priority access to the specified CPU and memory resources.
[role="_abstract"]
You can create a pod with a quality of service (QoS) class of `Guaranteed` for high-performance workloads. Configuring a pod with a QoS class of `Guaranteed` ensures that the pod has priority access to the specified CPU and memory resources.

To create a pod with a QoS class of `Guaranteed`, you must apply the following specifications:

Expand All @@ -27,11 +28,13 @@ In general, a pod with a QoS class of `Guaranteed` will not be evicted from a no
+
[source,terminal]
----
$ oc create namespace qos-example <1>
$ oc create namespace qos-example
----
<1> This example uses the `qos-example` namespace.
+
.Example output
This example uses the `qos-example` namespace.
+
The output resembles the following example:
+
[source,terminal]
----
namespace/qos-example created
Expand All @@ -42,7 +45,8 @@ namespace/qos-example created
.. Create a YAML file that defines the `Pod` resource:
+
--
.Example `qos-example.yaml` file
Create a `qos-example.yaml` file as in the following example:

[source,yaml]
----
apiVersion: v1
Expand All @@ -57,24 +61,25 @@ spec:
type: RuntimeDefault
containers:
- name: qos-demo-ctr
image: quay.io/openshifttest/hello-openshift:openshift <1>
image: quay.io/openshifttest/hello-openshift:openshift
resources:
limits:
memory: "200Mi" <2>
cpu: "1" <3>
memory: "200Mi"
cpu: "1"
requests:
memory: "200Mi" <4>
cpu: "1" <5>
memory: "200Mi"
cpu: "1"
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: [ALL]
----
<1> This example uses a public `hello-openshift` image.
<2> Sets the memory limit to 200 MB.
<3> Sets the CPU limit to 1 CPU.
<4> Sets the memory request to 200 MB.
<5> Sets the CPU request to 1 CPU.

* `image` specifies a public `hello-openshift` image.
* `limits.memory` sets the memory limit to 200 MB.
* `limits.cpu` sets the CPU limit to 1 CPU.
* `requests.memory` sets the memory request to 200 MB.
* `requests.cpu` sets the CPU request to 1 CPU.
+
[NOTE]
====
Expand All @@ -89,7 +94,8 @@ If you specify a memory limit for a container, but do not specify a memory reque
$ oc apply -f qos-example.yaml --namespace=qos-example
----
+
.Example output
The output resembles the following example:
+
[source,terminal]
----
pod/qos-demo created
Expand All @@ -104,7 +110,8 @@ pod/qos-demo created
$ oc get pod qos-demo --namespace=qos-example --output=yaml | grep qosClass
----
+
.Example output
The output resembles the following example:
+
[source,yaml]
----
qosClass: Guaranteed
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,11 @@
//
// * scalability_and_performance/low_latency_tuning/cnf-provisioning-low-latency-workloads.adoc

:_mod-docs-content-type: PROCEDURE
:_mod-docs-content-type: CONCEPT
[id="cnf-node-tuning-operator-disabling-cpu-load-balancing-for-dpdk_{context}"]
= Disabling CPU load balancing in a Pod

[role="_abstract"]
Functionality to disable or enable CPU load balancing is implemented on the CRI-O level. The code under the CRI-O disables or enables CPU load balancing only when the following requirements are met.

* The pod must use the `performance-<profile-name>` runtime class. You can get the proper name by looking at the status of the performance profile, as shown here:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
[id="cnf-scheduling-workload-onto-worker-with-real-time-capabilities_{context}"]
= Scheduling a low latency workload onto a worker with real-time capabilities

[role="_abstract"]
You can schedule low latency workloads onto a worker node where a performance profile that configures real-time capabilities is applied.

[NOTE]
Expand Down Expand Up @@ -34,9 +35,9 @@ kind: Pod
metadata:
name: dynamic-low-latency-pod
annotations:
cpu-quota.crio.io: "disable" <1>
cpu-load-balancing.crio.io: "disable" <2>
irq-load-balancing.crio.io: "disable" <3>
cpu-quota.crio.io: "disable"
cpu-load-balancing.crio.io: "disable"
irq-load-balancing.crio.io: "disable"
spec:
securityContext:
runAsNonRoot: true
Expand All @@ -58,15 +59,17 @@ spec:
capabilities:
drop: [ALL]
nodeSelector:
node-role.kubernetes.io/worker-cnf: "" <4>
runtimeClassName: performance-dynamic-low-latency-profile <5>
node-role.kubernetes.io/worker-cnf: ""
runtimeClassName: performance-dynamic-low-latency-profile
# ...
----
<1> Disables the CPU completely fair scheduler (CFS) quota at the pod run time.
<2> Disables CPU load balancing.
<3> Opts the pod out of interrupt handling on the node.
<4> The `nodeSelector` label must match the label that you specify in the `Node` CR.
<5> `runtimeClassName` must match the name of the performance profile configured in the cluster.
where:

`cpu-quota.crio.io: "disable"`:: Disables the CPU completely fair scheduler (CFS) quota at the pod run time.
`cpu-load-balancing.crio.io: "disable"`:: Disables CPU load balancing.
`irq-load-balancing.crio.io: "disable"`:: Opts the pod out of interrupt handling on the node.
`node-role.kubernetes.io/worker-cnf: ""`:: The `nodeSelector` label must match the label that you specify in the `Node` CR.
`performance-dynamic-low-latency-profile`:: The `runtimeClassName` must match the name of the performance profile configured in the cluster.

. Enter the pod `runtimeClassName` in the form performance-<profile_name>, where <profile_name> is the `name` from the `PerformanceProfile` YAML.
In the previous example, the `name` is `performance-dynamic-low-latency-profile`.
Expand All @@ -78,7 +81,7 @@ In the previous example, the `name` is `performance-dynamic-low-latency-profile`
$ oc get pod -o wide
----
+
.Expected output
The following output is expected:
+
[source,terminal]
----
Expand All @@ -93,7 +96,7 @@ dynamic-low-latency-pod 1/1 Running 0 5h33m 10.131.0.10 cnf-w
$ oc exec -it dynamic-low-latency-pod -- /bin/bash -c "grep Cpus_allowed_list /proc/self/status | awk '{print $2}'"
----
+
.Expected output
The following output is expected:
+
[source,terminal]
----
Expand All @@ -118,7 +121,7 @@ $ oc debug node/<node-name>
sh-4.4# chroot /host
----
+
.Expected output
The following output is expected:
+
[source,terminal]
----
Expand All @@ -132,7 +135,7 @@ sh-4.4#
sh-4.4# cat /proc/irq/default_smp_affinity
----
+
.Example output
The output resembles the following example:
+
[source,terminal]
----
Expand All @@ -146,7 +149,7 @@ sh-4.4# cat /proc/irq/default_smp_affinity
sh-4.4# find /proc/irq/ -name smp_affinity_list -exec sh -c 'i="$1"; mask=$(cat $i); file=$(echo $i); echo $file: $mask' _ {} \;
----
+
.Example output
The output resembles the following example:
+
[source,terminal]
----
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ include::modules/cnf-configuring-high-priority-workload-pods.adoc[leveloffset=+1
[role="_additional-resources"]
.Additional resources

xref:../scalability_and_performance/cnf-tuning-low-latency-nodes-with-perf-profile.adoc#cnf-configuring-power-saving-for-nodes_cnf-low-latency-perf-profile[Configuring power saving for nodes that run colocated high and low priority workloads]
* xref:../scalability_and_performance/cnf-tuning-low-latency-nodes-with-perf-profile.adoc#cnf-configuring-power-saving-for-nodes_cnf-low-latency-perf-profile[Configuring power saving for nodes that run colocated high and low priority workloads]

include::modules/cnf-disabling-cpu-cfs-quota.adoc[leveloffset=+1]

Expand Down