Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 47 additions & 44 deletions modules/nw-configure-sysctl-interface-sriov-network-bonded.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
[id="configuring-sysctl-on-bonded-sriov-network_{context}"]
= Configuring sysctl on a bonded SR-IOV network

[role="_abstract"]
You can set interface specific `sysctl` settings on a bonded interface created from two SR-IOV interfaces. Do this by adding the tuning configuration to the optional `Plugins` parameter of the bond network attachment definition.

[NOTE]
Expand All @@ -29,18 +30,21 @@ To change specific interface-level network `sysctl` settings create the `SriovNe
apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetwork
metadata:
name: allvalidflags <1>
namespace: openshift-sriov-network-operator <2>
name: allvalidflags
namespace: openshift-sriov-network-operator
spec:
resourceName: policyallflags <3>
networkNamespace: sysctl-tuning-test <4>
capabilities: '{ "mac": true, "ips": true }' <5>
resourceName: policyallflags
networkNamespace: sysctl-tuning-test
capabilities: '{ "mac": true, "ips": true }'
----
<1> A name for the object. The SR-IOV Network Operator creates a NetworkAttachmentDefinition object with same name.
<2> The namespace where the SR-IOV Network Operator is installed.
<3> The value for the `spec.resourceName` parameter from the `SriovNetworkNodePolicy` object that defines the SR-IOV hardware for this additional network.
<4> The target namespace for the `SriovNetwork` object. Only pods in the target namespace can attach to the additional network.
<5> Optional: The capabilities to configure for this additional network. You can specify `"{ "ips": true }"` to enable IP address support or `"{ "mac": true }"` to enable MAC address support.
+
--
* `<name>`: A name for the object. The SR-IOV Network Operator creates a NetworkAttachmentDefinition object with same name.
* `<namespace>`: The namespace where the SR-IOV Network Operator is installed.
* `<resourceName>`: The value for the `spec.resourceName` parameter from the `SriovNetworkNodePolicy` object that defines the SR-IOV hardware for this additional network.
* `<networkNamespace>`: The target namespace for the `SriovNetwork` object. Only pods in the target namespace can attach to the additional network.
* `<capabilities>`: Optional: The capabilities to configure for this additional network. You can specify `"{ "ips": true }"` to enable IP address support or `"{ "mac": true }"` to enable MAC address support.
--

. Create the `SriovNetwork` resource:
+
Expand All @@ -64,21 +68,21 @@ spec:
"name":"bound-net",
"plugins":[
{
"type":"bond", <1>
"mode": "active-backup", <2>
"failOverMac": 1, <3>
"linksInContainer": true, <4>
"type":"bond",
"mode": "active-backup",
"failOverMac": 1,
"linksInContainer": true,
"miimon": "100",
"links": [ <5>
"links": [
{"name": "net1"},
{"name": "net2"}
],
"ipam":{ <6>
"ipam":{
"type":"static"
}
},
{
"type":"tuning", <7>
"type":"tuning",
"capabilities":{
"mac":true
},
Expand All @@ -97,19 +101,16 @@ spec:
]
}'
----
<1> The type is `bond`.
<2> The `mode` attribute specifies the bonding mode. The bonding modes supported are:

* `balance-rr` - 0
* `active-backup` - 1
* `balance-xor` - 2
+
For `balance-rr` or `balance-xor` modes, you must set the `trust` mode to `on` for the SR-IOV virtual function.
<3> The `failover` attribute is mandatory for active-backup mode.
<4> The `linksInContainer=true` flag informs the Bond CNI that the required interfaces are to be found inside the container. By default, Bond CNI looks for these interfaces on the host which does not work for integration with SRIOV and Multus.
<5> The `links` section defines which interfaces will be used to create the bond. By default, Multus names the attached interfaces as: "net", plus a consecutive number, starting with one.
<6> A configuration object for the IPAM CNI plugin as a YAML block scalar. The plugin manages IP address assignment for the attachment definition. In this pod example IP addresses are configured manually, so in this case,`ipam` is set to static.
<7> Add additional capabilities to the device. For example, set the `type` field to `tuning`. Specify the interface-level network `sysctl` you want to set in the sysctl field. This example sets all interface-level network `sysctl` settings that can be set.
--
* `<type>`: The type is `bond`.
* `<mode>`: The `mode` attribute specifies the bonding mode. The bonding modes supported are `balance-rr` - 0, `active-backup` - 1, and `balance-xor` - 2. For `balance-rr` or `balance-xor` modes, you must set the `trust` mode to `on` for the SR-IOV virtual function.
* `<failOverMac>`: The `failover` attribute is mandatory for active-backup mode.
* `<linksInContainer>`: The `linksInContainer=true` flag informs the Bond CNI that the required interfaces are to be found inside the container. By default, Bond CNI looks for these interfaces on the host which does not work for integration with SRIOV and Multus.
* `<links>`: The `links` section defines which interfaces will be used to create the bond. By default, Multus names the attached interfaces as: "net", plus a consecutive number, starting with one.
* `<ipam>`: A configuration object for the IPAM CNI plugin as a YAML block scalar. The plugin manages IP address assignment for the attachment definition. In this pod example IP addresses are configured manually, so in this case,`ipam` is set to static.
* `<tuning>`: Add additional capabilities to the device. For example, set the `type` field to `tuning`. Specify the interface-level network `sysctl` you want to set in the sysctl field. This example sets all interface-level network `sysctl` settings that can be set.
--

. Create the bond network attachment resource:
+
Expand All @@ -118,25 +119,24 @@ For `balance-rr` or `balance-xor` modes, you must set the `trust` mode to `on` f
$ oc create -f sriov-bond-network-interface.yaml
----

.Verifying that the `NetworkAttachmentDefinition` CR is successfully created
.Verification

* Confirm that the SR-IOV Network Operator created the `NetworkAttachmentDefinition` CR by running the following command:
. Confirm that the SR-IOV Network Operator created the `NetworkAttachmentDefinition` CR by running the following command:
+
[source,terminal]
----
$ oc get network-attachment-definitions -n <namespace> <1>
$ oc get network-attachment-definitions -n <namespace>
----
<1> Replace `<namespace>` with the networkNamespace that you specified when configuring the network attachment, for example, `sysctl-tuning-test`. Expected output shows the names of the NAD CRDs and the creation age in minutes.
+
--
* `<namespace>`: Replace with the networkNamespace that you specified when configuring the network attachment, for example, `sysctl-tuning-test`. Expected output shows the names of the NAD CRDs and the creation age in minutes.
--
+
[NOTE]
====
There might be a delay before the SR-IOV Network Operator creates the CR.
====

.Verifying that the additional SR-IOV network resource is successful

To verify that the tuning CNI is correctly configured and the additional SR-IOV network attachment is attached, do the following:

. Create a `Pod` CR. For example, save the following YAML as the file `examplepod.yaml`:
+
[source,yaml]
Expand All @@ -149,13 +149,13 @@ metadata:
annotations:
k8s.v1.cni.cncf.io/networks: |-
[
{"name": "allvalidflags"}, <1>
{"name": "allvalidflags"},
{"name": "allvalidflags"},
{
"name": "bond-sysctl-network",
"interface": "bond0",
"mac": "0a:56:0a:83:04:0c", <2>
"ips": ["10.100.100.200/24"] <3>
"mac": "0a:56:0a:83:04:0c",
"ips": ["10.100.100.200/24"]
}
]
spec:
Expand All @@ -174,9 +174,12 @@ spec:
seccompProfile:
type: RuntimeDefault
----
<1> The name of the SR-IOV network attachment definition CR.
<2> Optional: The MAC address for the SR-IOV device that is allocated from the resource type defined in the SR-IOV network attachment definition CR. To use this feature, you also must specify `{ "mac": true }` in the SriovNetwork object.
<3> Optional: IP addresses for the SR-IOV device that are allocated from the resource type defined in the SR-IOV network attachment definition CR. Both IPv4 and IPv6 addresses are supported. To use this feature, you also must specify `{ "ips": true }` in the `SriovNetwork` object.
+
--
* `<allvalidflags>`: The name of the SR-IOV network attachment definition CR.
* `<mac>`: Optional: The MAC address for the SR-IOV device that is allocated from the resource type defined in the SR-IOV network attachment definition CR. To use this feature, you also must specify `{ "mac": true }` in the SriovNetwork object.
* `<ips>`: Optional: IP addresses for the SR-IOV device that are allocated from the resource type defined in the SR-IOV network attachment definition CR. Both IPv4 and IPv6 addresses are supported. To use this feature, you also must specify `{ "ips": true }` in the `SriovNetwork` object.
--

. Apply the YAML:
+
Expand All @@ -192,7 +195,7 @@ $ oc apply -f examplepod.yaml
$ oc get pod -n sysctl-tuning-test
----
+
.Example output
The following is example output:
+
[source,terminal]
----
Expand Down
64 changes: 35 additions & 29 deletions modules/nw-configure-sysctl-interface-sriov-network.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
[id="configuring-sysctl-on-sriov-network_{context}"]
= Configuring sysctl on a SR-IOV network

[role="_abstract"]
You can set interface specific `sysctl` settings on virtual interfaces created by SR-IOV by adding the tuning configuration to the optional `metaPlugins` parameter of the `SriovNetwork` resource.

The SR-IOV Network Operator manages additional network definitions. When you specify an additional SR-IOV network to create, the SR-IOV Network Operator creates the `NetworkAttachmentDefinition` custom resource (CR) automatically.
Expand All @@ -31,14 +32,14 @@ To change the interface-level network `net.ipv4.conf.IFNAME.accept_redirects` `s
apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetwork
metadata:
name: onevalidflag <1>
namespace: openshift-sriov-network-operator <2>
name: onevalidflag
namespace: openshift-sriov-network-operator
spec:
resourceName: policyoneflag <3>
networkNamespace: sysctl-tuning-test <4>
ipam: '{ "type": "static" }' <5>
capabilities: '{ "mac": true, "ips": true }' <6>
metaPlugins : | <7>
resourceName: policyoneflag
networkNamespace: sysctl-tuning-test
ipam: '{ "type": "static" }'
capabilities: '{ "mac": true, "ips": true }'
metaPlugins : |
{
"type": "tuning",
"capabilities":{
Expand All @@ -49,13 +50,16 @@ spec:
}
}
----
<1> A name for the object. The SR-IOV Network Operator creates a NetworkAttachmentDefinition object with same name.
<2> The namespace where the SR-IOV Network Operator is installed.
<3> The value for the `spec.resourceName` parameter from the `SriovNetworkNodePolicy` object that defines the SR-IOV hardware for this additional network.
<4> The target namespace for the `SriovNetwork` object. Only pods in the target namespace can attach to the additional network.
<5> A configuration object for the IPAM CNI plugin as a YAML block scalar. The plugin manages IP address assignment for the attachment definition.
<6> Optional: Set capabilities for the additional network. You can specify `"{ "ips": true }"` to enable IP address support or `"{ "mac": true }"` to enable MAC address support.
<7> Optional: The metaPlugins parameter is used to add additional capabilities to the device. In this use case set the `type` field to `tuning`. Specify the interface-level network `sysctl` you want to set in the `sysctl` field.
+
--
* `<name>` specifies a name for the object. The SR-IOV Network Operator creates a NetworkAttachmentDefinition object with same name.
* `<namespace>` specifies the namespace where the SR-IOV Network Operator is installed.
* `<resourceName>` specifies the value for the `spec.resourceName` parameter from the `SriovNetworkNodePolicy` object that defines the SR-IOV hardware for this additional network.
* `<networkNamespace>` specifies the target namespace for the `SriovNetwork` object. Only pods in the target namespace can attach to the additional network.
* `<ipam>` specifies a configuration object for the IPAM CNI plugin as a YAML block scalar. The plugin manages IP address assignment for the attachment definition.
* `<capabilities>` specifies optional capabilities for the additional network. You can specify `"{ "ips": true }"` to enable IP address support or `"{ "mac": true }"` to enable MAC address support.
* `<metaPlugins>` specifies optional additional capabilities for the device. In this use case set the `type` field to `tuning`. Specify the interface-level network `sysctl` you want to set in the `sysctl` field.
--

. Create the `SriovNetwork` resource:
+
Expand All @@ -64,25 +68,24 @@ spec:
$ oc create -f sriov-network-interface-sysctl.yaml
----

.Verifying that the `NetworkAttachmentDefinition` CR is successfully created
.Verification

* Confirm that the SR-IOV Network Operator created the `NetworkAttachmentDefinition` CR by running the following command:
+
[source,terminal]
----
$ oc get network-attachment-definitions -n <namespace> <1>
$ oc get network-attachment-definitions -n <namespace>
----
<1> Replace `<namespace>` with the value for `networkNamespace` that you specified in the `SriovNetwork` object. For example, `sysctl-tuning-test`. The expected output shows the name of the NAD CRD and the creation age in minutes.
+
--
* Replace `<namespace>` with the value for `networkNamespace` that you specified in the `SriovNetwork` object. For example, `sysctl-tuning-test`. The expected output shows the name of the NAD CRD and the creation age in minutes.
--
+
[NOTE]
====
There might be a delay before the SR-IOV Network Operator creates the CR.
====

.Verifying that the additional SR-IOV network attachment is successful

To verify that the tuning CNI is correctly configured and the additional SR-IOV network attachment is attached, do the following:

. Create a `Pod` CR. Save the following YAML as the file `examplepod.yaml`:
+
[source,yaml]
Expand All @@ -96,9 +99,9 @@ metadata:
k8s.v1.cni.cncf.io/networks: |-
[
{
"name": "onevalidflag", <1>
"mac": "0a:56:0a:83:04:0c", <2>
"ips": ["10.100.100.200/24"] <3>
"name": "onevalidflag",
"mac": "0a:56:0a:83:04:0c",
"ips": ["10.100.100.200/24"]
}
]
spec:
Expand All @@ -117,9 +120,12 @@ spec:
seccompProfile:
type: RuntimeDefault
----
<1> The name of the SR-IOV network attachment definition CR.
<2> Optional: The MAC address for the SR-IOV device that is allocated from the resource type defined in the SR-IOV network attachment definition CR. To use this feature, you also must specify `{ "mac": true }` in the SriovNetwork object.
<3> Optional: IP addresses for the SR-IOV device that are allocated from the resource type defined in the SR-IOV network attachment definition CR. Both IPv4 and IPv6 addresses are supported. To use this feature, you also must specify `{ "ips": true }` in the `SriovNetwork` object.
+
--
* `<name>` specifies the name of the SR-IOV network attachment definition CR.
* `<mac>` is optional. The MAC address for the SR-IOV device that is allocated from the resource type defined in the SR-IOV network attachment definition CR. To use this feature, you also must specify `{ "mac": true }` in the SriovNetwork object.
* `<ips>` is optional. IP addresses for the SR-IOV device that are allocated from the resource type defined in the SR-IOV network attachment definition CR. Both IPv4 and IPv6 addresses are supported. To use this feature, you also must specify `{ "ips": true }` in the `SriovNetwork` object.
--

. Create the `Pod` CR:
+
Expand All @@ -135,7 +141,7 @@ $ oc apply -f examplepod.yaml
$ oc get pod -n sysctl-tuning-test
----
+
.Example output
The following is example output:
+
[source,terminal]
----
Expand All @@ -150,7 +156,7 @@ tunepod 1/1 Running 0 47s
$ oc rsh -n sysctl-tuning-test tunepod
----

. Verify the values of the configured sysctl flag. Find the value `net.ipv4.conf.IFNAME.accept_redirects` by running the following command::
. Verify the values of the configured sysctl flag. Find the value `net.ipv4.conf.IFNAME.accept_redirects` by running the following command:
+
[source,terminal]
----
Expand Down
Loading