kubernetes cloud-init problem

[PPisz]

Hello,

When I start a kubernetes cluster in CS it generally starts, I can download kubeconfig and log into the cluster. However, if I look at the node console, I see that there is an error:

Failed to start Execute cloud/user final script

The clod-init.log contains the following information:

2025-02-10 12:53:31,110 - handlers.py[DEBUG]: start: modules-final/config-scripts-user: running config-scripts-user with frequency once-per-instance 2025-02-10 12:53:31,110 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/c8e318d8-03ca-41d7-9104-1f9572e5103e/sem/config_scripts_user - wb: [644] 25 bytes 2025-02-10 12:53:31,110 - helpers.py[DEBUG]: Running config-scripts-user using lock (<FileLock using file '/var/lib/cloud/instances/c8e318d8-03ca-41d7-9104-1f9572e5103e/sem/config_scripts_user'>) 2025-02-10 12:53:31,110 - subp.py[DEBUG]: Running command ['/var/lib/cloud/instance/scripts/runcmd'] with allowed return codes [0] (shell=False, capture=False) 2025-02-10 12:55:14,903 - subp.py[DEBUG]: Unexpected error while running command. Command: ['/var/lib/cloud/instance/scripts/runcmd'] Exit code: 1 Reason: - Stdout: - Stderr: - 2025-02-10 12:55:14,905 - cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts) 2025-02-10 12:55:14,906 - handlers.py[DEBUG]: finish: modules-final/config-scripts-user: FAIL: running config-scripts-user with frequency once-per-instance 2025-02-10 12:55:14,907 - util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_scripts_user.py'>) failed 2025-02-10 12:55:14,907 - util.py[DEBUG]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_scripts_user.py'>) failed Traceback (most recent call last): File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 848, in _run_modules ran, _r = cc.run(run_name, mod.handle, func_args, File "/usr/lib/python3/dist-packages/cloudinit/cloud.py", line 54, in run return self._runners.run(name, functor, args, freq, clear_on_fail) File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 185, in run results = functor(*args) File "/usr/lib/python3/dist-packages/cloudinit/config/cc_scripts_user.py", line 45, in handle subp.runparts(runparts_path) File "/usr/lib/python3/dist-packages/cloudinit/subp.py", line 384, in runparts raise RuntimeError( RuntimeError: Runparts: 1 failures (runcmd) in 1 attempted commands 2025-02-10 12:55:14,919 - stages.py[DEBUG]: Running module ssh-authkey-fingerprints (<module 'cloudinit.config.cc_ssh_authkey_fingerprints' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_ssh_authkey_fingerprints.py'>) with frequency once-per-instance 2025-02-10 12:55:14,920 - handlers.py[DEBUG]: start: modules-final/config-ssh-authkey-fingerprints: running config-ssh-authkey-fingerprints with frequency once-per-instance 2025-02-10 12:55:14,921 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/c8e318d8-03ca-41d7-9104-1f9572e5103e/sem/config_ssh_authkey_fingerprints - wb: [644] 24 bytes 2025-02-10 12:55:14,922 - helpers.py[DEBUG]: Running config-ssh-authkey-fingerprints using lock (<FileLock using file '/var/lib/cloud/instances/c8e318d8-03ca-41d7-9104-1f9572e5103e/sem/config_ssh_authkey_fingerprints'>)

This happens to me on all versions of kubernetes and is very disturbing. What could be causing this?

[Pearl1594]

Are you able to log into the nodes via SSH using:
ssh -i <ssh key that you injected into the cluster / Management server private key> -p 2222 cloud@<public_ip> <-- to log into the control node?

[PPisz]

Yes, I can log (both keys) in to the control node and all worker nodes. The problem never occurs on the control node, always on the worker nodes

[Pearl1594]

Do the clusters come up? Because, the error on the console wrt failing to run cloud/user final script is an expected behaviour. As the worker nodes are constantly trying to connect with the control node, so until the control node is up, the script (deploy-kube-system) may fail initially.

[PPisz]

Ok thanks for the explanation, yes the cluster is UP and fully functional. I wanted to make sure :-)