Support vertical scaling of Control Plane nodes

[rcheeran]

Title:Vertical scaling of control plane nodes

Description:
In 22h2 we could vertically scale (change the cpu/memory) of the control plane using Set-AksHciCluster: https://msazure.visualstudio.com/msk8s/_git/cloudmanager?path=/script/AksHci/AksHci.psm1&version=GBmaster&line=1871&lineEnd=1872&lineStartColumn=1&lineEndColumn=1&lineStyle=plain&_a=contents

However, that ability seems absent from az aksarc update.

We should add that back for 23h2.

[Elektronenvolt]

Hi @rcheeran

The top top top feature on our Wishlist! - wanted to type a feature request right now. We have more and more situations where control planes run out of CPU during a spike of incoming jobs e.g. My current workaround is manually change CPU cores over Hyper-V tools - shutdown - change fast to not trigger a recovery action - start up - control plane node by node. If there is a better workaround, please let us know. The other option is to create a new cluster and move over workloads, but that's not always that easy.

[rcheeran]

[like] Roycey Cheeran reacted to your message:

…

________________________________ From: Georg Stoifl ***@***.***> Sent: Friday, August 8, 2025 8:44:19 PM To: Azure/aksArc ***@***.***> Cc: Mention ***@***.***>; Author ***@***.***>; Assign ***@***.***> Subject: Re: [Azure/aksArc] Support vertical scaling of Control Plane nodes (Issue #414) Hi @rcheeran<https://github.com/rcheeran> The top top top feature on our Wishlist! - wanted to type a feature request right now. We have more and more situations where control planes run out of CPU during a spike of incoming jobs e.g. My current workaround is manually change CPU cores over Hyper-V tools - shutdown - change fast to not trigger a recovery action - start up - control plane node by node. If there is a better workaround, please let us know. The other option is to create a new cluster and move over workloads, but that's not always that easy. — Reply to this email directly, view it on GitHub<#414 (comment)> or unsubscribe<https://github.com/notifications/unsubscribe-auth/AYXFYVKGS6E42MJZN4MP5EL3MUD2LBFKMF2HI4TJMJ2XIZLTSSBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLAVFOZQWY5LFVI2TCNRQGEZDQOJTGWSG4YLNMWUWQYLTL5WGCYTFNSWHG5LCNJSWG5C7OR4XAZNMJFZXG5LFINXW23LFNZ2KM5DPOBUWG44TQKSHI6LQMWVHEZLQN5ZWS5DPOJ42K5TBNR2WLKJSHEYDQMZUHEYTHAVEOR4XAZNFNFZXG5LFUV3GC3DVMWVDGMZQGEYTQMBYG4YIFJDUPFYGLJLMMFRGK3FFOZQWY5LFVI2TCNRQGEZDQOJTGWTXI4TJM5TWK4VGMNZGKYLUMU>. You are receiving this email because you were mentioned. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[hamelchRIT]

push up... --> VOTE UP

[Elektronenvolt]

@hamelchRIT - I'm "oversizing" control planes at new clusters as workaround. In case you need to scale - doing it over Hyper-tools / Failover Cluster Manager worked fine. Shutdown - change cores and mem, start up. It's a short outage. In the case of multiple control planes, I did a coffee break between each node to maintain quorum.

[hamelchRIT]

@Elektronenvolt
Yes we can do many things in the background, but those things are unsupported thing, what can result in inconsistency or maybe resulting in deployment issues if you're deploying from / into deploying stacks etc...

those things must be configurable over cmdlets or portal... and not by manipulating the hypervisor on the physical machine.
also for future releases, that can consume more resources... we don't want to recreate / migrate a whole cluster and workload to a new one, when we can perform "online" maschine-scaling for nodes, this is also a must feature on the control-plane level

also if you try to change the node annotations... yes you can do this... but they are automatically overridden / overwritten...
same if you change some deployments directly inside the K8s they are enforced to the origin configuration...

i can't tell you when this is happen, but i can tell you that's done, because i've tried this once to save some memory and cpu from some K8s-Deployments created by MS-Charts...

[Elektronenvolt]

Yes - "don't do this at home" flag on it. It's a workaround we used until the feature is available. We only did that on a few clusters to avoid a re-deployment. In urgent cases, that works, but yes - try to avoid it, and create a new cluster with bigger control plane VM sizes instead.

[rjaini]

We are planning to get this feature out soon in 2026.