Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
/bin/

# Forge workflow state (do not commit)
.forge/
5 changes: 5 additions & 0 deletions data/data/install.openshift.io_installconfigs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7428,6 +7428,11 @@ spec:
type: string
maxItems: 2
type: array
bootstrapFlavor:
description: |-
BootstrapFlavor is the name of the flavor used for the bootstrap instance.
When not specified, the bootstrap machine will use the control plane flavor.
type: string
cloud:
description: Cloud is the name of OpenStack cloud to use from
clouds.yaml.
Expand Down
39 changes: 38 additions & 1 deletion docs/user/openstack/customization.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Beyond the [platform-agnostic `install-config.yaml` properties](../customization
- [Examples](#examples)
- [Minimal](#minimal)
- [Custom machine pools](#custom-machine-pools)
- [Bootstrap flavor for NFV/NUMA workloads](#bootstrap-flavor-for-nfvnuma-workloads)
- [Image Overrides](#image-overrides)
- [Custom Subnets](#custom-subnets)
- [Additional Networks](#additional-networks)
Expand All @@ -34,6 +35,7 @@ Beyond the [platform-agnostic `install-config.yaml` properties](../customization
* `apiVIPs` (optional array of strings): IP address on the machineNetwork that will be assigned to the API VIP. If more than one are set, it must be one IPv4 and one IPv6.
* `ingressVIPs` (optional array of strings): IP address on the machineNetwork that will be assigned to the ingress VIP. If more than one are set, it must be one IPv4 and one IPv6.
* `controlPlanePort` (optional object): the UUID and/or Name of an OpenStack Network and its Subnets where to install the nodes of the cluster onto. For more information on how to install with a custom subnet, see the [custom subnets](#custom-subnets) section of the docs.
* `bootstrapFlavor` (optional string): The OpenStack flavor to use for the temporary bootstrap instance. When not set, the bootstrap instance uses the same flavor as the control plane machines. This is useful in environments where the control plane flavor has special hardware requirements (such as NFV/NUMA pinning) that are incompatible with the general-purpose bootstrap workload — in those cases a standard flavor can be specified here. See [Bootstrap flavor for NFV/NUMA workloads](#bootstrap-flavor-for-nfvnuma-workloads) for details.
* `defaultMachinePlatform` (optional object): Default [OpenStack-specific machine pool properties](#machine-pools) which apply to [machine pools](../customization.md#machine-pools) that do not define their own OpenStack-specific properties.

## Machine pools
Expand All @@ -56,7 +58,7 @@ Beyond the [platform-agnostic `install-config.yaml` properties](../customization
* `zones` (optional list of strings): The names of the availability zones you want to install your nodes on. If unset, the installer will use your default compute zone.

> **Note**
> The bootstrap node follows the `type`, `rootVolume`, `additionalNetworkIDs`, and `additionalSecurityGroupIDs` parameters from the `controlPlane` machine pool.
> The bootstrap node follows the `type`, `rootVolume`, `additionalNetworkIDs`, and `additionalSecurityGroupIDs` parameters from the `controlPlane` machine pool. The bootstrap flavor can be overridden independently using the cluster-scoped `bootstrapFlavor` property.

> **Note**
> Note when deploying the control-plane machines with `rootVolume`, it is highly suggested to use an [additional ephemeral disk dedicated to etcd](./etcd-ephemeral-disk.md).
Expand Down Expand Up @@ -122,6 +124,41 @@ pullSecret: '{"auths": ...}'
sshKey: ssh-ed25519 AAAA...
```

### Bootstrap flavor for NFV/NUMA workloads

In Network Function Virtualization (NFV) environments, control plane machines are often deployed on flavors with strict NUMA topology policies or CPU pinning to meet latency and throughput requirements. Because the bootstrap node is a temporary machine that runs a standard Kubernetes control plane (not an NFV workload), deploying it on an NFV-optimized flavor can cause scheduling or compatibility issues — for example, flavors with `hw:numa_nodes` or CPU pinning extra specs may not be schedulable on all hypervisor hosts.

The `bootstrapFlavor` field lets you decouple the bootstrap instance from the control plane flavor. When set, the bootstrap instance uses the specified flavor; when omitted, it falls back to the flavor defined in the `controlPlane` machine pool (or `defaultMachinePlatform`).

Example — using a standard flavor for the bootstrap node while the control plane uses an NFV/NUMA-optimized flavor:

```yaml
apiVersion: v1
baseDomain: example.com
metadata:
name: test-cluster
controlPlane:
name: master
replicas: 3
platform:
openstack:
type: nfv.numa.xlarge # NFV-optimized flavor for production control plane nodes
platform:
openstack:
cloud: mycloud
externalNetwork: external
apiFloatingIP: 128.0.0.1
bootstrapFlavor: m1.xlarge # Standard flavor for the temporary bootstrap instance
pullSecret: '{"auths": ...}'
sshKey: ssh-ed25519 AAAA...
```

> **Note**
> The bootstrap instance is temporary and is removed once the cluster is fully operational. The `bootstrapFlavor` has no effect on control plane or worker machine flavors.

> **Note**
> If `bootstrapFlavor` is not set, the bootstrap instance uses the flavor from the `controlPlane` machine pool. If that is also unset, it falls back to `defaultMachinePlatform.type`.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

## Image Overrides

The OpenShift installer pins the version of RHEL CoreOS and normally handles uploading the image to the target OpenStack instance.
Expand Down
13 changes: 13 additions & 0 deletions pkg/asset/installconfig/openstack/validation/cloudinfo.go
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,19 @@ func (ci *CloudInfo) collectInfo(ctx context.Context, ic *types.InstallConfig) e
}
}
}

// Get bootstrap flavor info if specified
if flavorName := ic.OpenStack.BootstrapFlavor; flavorName != "" {
if _, seen := ci.Flavors[flavorName]; !seen {
flavor, err := ci.getFlavor(ctx, flavorName)
if !gophercloud.ResponseCodeIs(err, http.StatusNotFound) {
if err != nil {
return err
}
ci.Flavors[flavorName] = flavor
}
}
}
if ic.OpenStack.ControlPlanePort != nil {
controlPlanePort := ic.OpenStack.ControlPlanePort
ci.ControlPlanePortSubnets, err = ci.getSubnets(ctx, controlPlanePort)
Expand Down
16 changes: 16 additions & 0 deletions pkg/asset/installconfig/openstack/validation/platform.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,9 @@ func ValidatePlatform(p *openstack.Platform, n *types.Networking, ci *CloudInfo)
// validate custom cluster os image
allErrs = append(allErrs, validateClusterOSImage(p, ci, fldPath)...)

// validate bootstrap flavor
allErrs = append(allErrs, validateBootstrapFlavor(p, ci, fldPath)...)

return allErrs
}

Expand Down Expand Up @@ -204,6 +207,19 @@ func validateVIPs(p *openstack.Platform, ci *CloudInfo, fldPath *field.Path) (al
return allErrs
}

// validateBootstrapFlavor validates the bootstrap flavor if specified, ensuring it exists in OpenStack.
func validateBootstrapFlavor(p *openstack.Platform, ci *CloudInfo, fldPath *field.Path) (allErrs field.ErrorList) {
if p.BootstrapFlavor == "" {
return
}

if _, ok := ci.Flavors[p.BootstrapFlavor]; !ok {
allErrs = append(allErrs, field.NotFound(fldPath.Child("bootstrapFlavor"), p.BootstrapFlavor))
}

return allErrs
}

// validateExternalNetwork validates the user's input for the clusterOSImage and returns a list of all validation errors
func validateClusterOSImage(p *openstack.Platform, ci *CloudInfo, fldPath *field.Path) (allErrs field.ErrorList) {
if p.ClusterOSImage == "" {
Expand Down
71 changes: 71 additions & 0 deletions pkg/asset/installconfig/openstack/validation/platform_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package validation
import (
"testing"

"github.com/gophercloud/gophercloud/v2/openstack/compute/v2/flavors"
"github.com/gophercloud/gophercloud/v2/openstack/image/v2/images"
"github.com/gophercloud/gophercloud/v2/openstack/networking/v2/extensions/layer3/floatingips"
"github.com/gophercloud/gophercloud/v2/openstack/networking/v2/extensions/mtu"
Expand Down Expand Up @@ -728,3 +729,73 @@ func TestMachineSubnet(t *testing.T) {
})
}
}

func TestBootstrapFlavor(t *testing.T) {
const (
validBootstrapFlavor = "valid-bootstrap-flavor"
invalidBootstrapFlavor = "invalid-bootstrap-flavor"
)

cases := []struct {
name string
platform *openstack.Platform
cloudInfo *CloudInfo
networking *types.Networking
expectedError bool
expectedErrMsg string // NOTE: this is a REGEXP
}{
{
name: "bootstrap flavor not specified",
platform: validPlatform(),
cloudInfo: validPlatformCloudInfo(),
networking: validNetworking(),
expectedError: false,
expectedErrMsg: "",
},
{
name: "valid bootstrap flavor",
platform: func() *openstack.Platform {
p := validPlatform()
p.BootstrapFlavor = validBootstrapFlavor
return p
}(),
cloudInfo: func() *CloudInfo {
ci := validPlatformCloudInfo()
ci.Flavors = map[string]Flavor{
validBootstrapFlavor: {
Flavor: flavors.Flavor{
Name: validBootstrapFlavor,
},
},
}
return ci
}(),
networking: validNetworking(),
expectedError: false,
expectedErrMsg: "",
},
{
name: "bootstrap flavor not found in OpenStack",
platform: func() *openstack.Platform {
p := validPlatform()
p.BootstrapFlavor = invalidBootstrapFlavor
return p
}(),
cloudInfo: validPlatformCloudInfo(),
networking: validNetworking(),
expectedError: true,
expectedErrMsg: `platform.openstack.bootstrapFlavor: Not found: "invalid-bootstrap-flavor"`,
},
}

for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
aggregatedErrors := ValidatePlatform(tc.platform, tc.networking, tc.cloudInfo).ToAggregate()
if tc.expectedError {
assert.Regexp(t, tc.expectedErrMsg, aggregatedErrors)
} else {
assert.NoError(t, aggregatedErrors)
}
})
}
}
10 changes: 9 additions & 1 deletion pkg/asset/machines/openstack/machines.go
Original file line number Diff line number Diff line change
Expand Up @@ -251,12 +251,20 @@ func generateProviderSpec(ctx context.Context, clusterID string, config *types.I
serverGroupName += "-" + failureDomain.AvailabilityZone
}

// Determine the flavor to use for this machine.
// Bootstrap machines use BootstrapFlavor when specified; otherwise fall back to the
// control plane flavor from the machine pool. Master machines always use FlavorName.
flavorName := mpool.FlavorName
if role == bootstrapRole && platform.BootstrapFlavor != "" {
flavorName = platform.BootstrapFlavor
}

spec := machinev1alpha1.OpenstackProviderSpec{
TypeMeta: metav1.TypeMeta{
APIVersion: machinev1alpha1.GroupVersion.String(),
Kind: "OpenstackProviderSpec",
},
Flavor: mpool.FlavorName,
Flavor: flavorName,
CloudName: CloudName,
CloudsSecret: &corev1.SecretReference{Name: cloudsSecret, Namespace: cloudsSecretNamespace},
UserDataSecret: &corev1.SecretReference{Name: userDataSecret},
Expand Down
79 changes: 79 additions & 0 deletions pkg/asset/machines/openstack/machines_test.go
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
package openstack

import (
"context"
"fmt"
"strings"
"testing"

machinev1 "github.com/openshift/api/machine/v1"
"github.com/openshift/installer/pkg/types"
"github.com/openshift/installer/pkg/types/openstack"
)

Expand Down Expand Up @@ -287,3 +289,80 @@ func TestFailureDomains(t *testing.T) {

func TestPruneFailureDomains(t *testing.T) {
}

// TestMAPIBootstrapFlavorSelection verifies that generateProviderSpec selects the correct
// flavor for bootstrap and master roles in the MAPI code path:
// - Bootstrap with BootstrapFlavor set → BootstrapFlavor is used
// - Bootstrap with BootstrapFlavor empty → control plane FlavorName is used (fallback)
// - Master with BootstrapFlavor set → FlavorName is used (master is unaffected)
func TestMAPIBootstrapFlavorSelection(t *testing.T) {
const (
controlPlaneFlavor = "m1.xlarge"
bootstrapFlavor = "m1.medium"
clusterID = "test-cluster"
osImage = "rhcos"
userDataSecret = "user-data"
)

emptyFD := machinev1.OpenStackFailureDomain{RootVolume: &machinev1.RootVolume{}}
configDrive := false

tests := []struct {
name string
role string
bootstrapFlavor string // value placed in platform.BootstrapFlavor
wantFlavor string
}{
{
name: "bootstrap uses BootstrapFlavor when specified",
role: bootstrapRole,
bootstrapFlavor: bootstrapFlavor,
wantFlavor: bootstrapFlavor,
},
{
name: "bootstrap falls back to control plane flavor when BootstrapFlavor is empty",
role: bootstrapRole,
bootstrapFlavor: "",
wantFlavor: controlPlaneFlavor,
},
{
name: "master always uses FlavorName regardless of BootstrapFlavor",
role: masterRole,
bootstrapFlavor: bootstrapFlavor,
wantFlavor: controlPlaneFlavor,
},
}

for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
config := &types.InstallConfig{
Platform: types.Platform{
OpenStack: &openstack.Platform{
BootstrapFlavor: tt.bootstrapFlavor,
},
},
}
mpool := &openstack.MachinePool{
FlavorName: controlPlaneFlavor,
}

spec, err := generateProviderSpec(
context.Background(),
clusterID,
config,
mpool,
osImage,
tt.role,
userDataSecret,
emptyFD,
&configDrive,
)
if err != nil {
t.Fatalf("generateProviderSpec() unexpected error: %v", err)
}
if got := spec.Flavor; got != tt.wantFlavor {
t.Errorf("Flavor = %q, want %q", got, tt.wantFlavor)
}
})
}
}
10 changes: 9 additions & 1 deletion pkg/asset/machines/openstack/openstackmachines.go
Original file line number Diff line number Diff line change
Expand Up @@ -195,8 +195,16 @@ func generateMachineSpec(clusterID string, config *types.InstallConfig, mpool *o
securityGroups = nil
}

// Determine the flavor to use for this machine.
// Bootstrap machines use BootstrapFlavor when specified; otherwise fall back to the
// control plane flavor from the machine pool. Master machines always use FlavorName.
flavorName := mpool.FlavorName
if role == bootstrapRole && platform.BootstrapFlavor != "" {
flavorName = platform.BootstrapFlavor
}

spec := capo.OpenStackMachineSpec{
Flavor: ptr.To(mpool.FlavorName),
Flavor: ptr.To(flavorName),
IdentityRef: &capo.OpenStackIdentityReference{
Name: clusterID + "-cloud-config",
CloudName: CloudName,
Expand Down
Loading