Follow up: test-macos10.15-x64-2 offline

[richardlau]

All of our macOS 10.14 VMs (all in Orka) are offline.

Initially it was because the agents needed to be updated after #3176 -- I've done that (via the upgrade-jar playbook), but now the agents are refusing to start as they need newer Java. It looks like these are still running Java 8.

[richardlau]

It also looks like test-orka-macos10.14-x64-2 is unreachable

test-orka-macos10.14-x64-1 : ok=3    changed=2    unreachable=0    failed=0    skipped=5    rescued=0    ignored=0
test-orka-macos10.14-x64-2 : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0
test-orka-macos10.14-x64-3 : ok=3    changed=2    unreachable=0    failed=0    skipped=4    rescued=0    ignored=0

[richardlau]

Looks like the orka 10.15 VMs are also offline. I logged into test-orka-macos10.15-x64-1 and that appears to be running Java 8 as well.

% java -version
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_242-b08)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.242-b08, mixed mode)

[richardlau]

These were updated to Java 17 previously #3030 (comment) / #3085.

[richardlau]

These were updated to Java 17 previously #3030 (comment) / #3085.

We rebuilt these VMs in January #3112.

[sxa]

Nearform macs are running with Eclipse Temurin 17 with /usr/bin/java pointing there.

administrator@test-nearform-macos10 ~iojs % java -version
openjdk version "17.0.6" 2023-01-17
OpenJDK Runtime Environment Temurin-17.0.6+10 (build 17.0.6+10)
OpenJDK 64-Bit Server VM Temurin-17.0.6+10 (build 17.0.6+10, mixed mode, sharing)
administrator@test-nearform-macos10 ~iojs % ls -l /usr/bin/java
lrwxr-xr-x  1 root  wheel  74 18 Dec 13:41 /usr/bin/java -> /System/Library/Frameworks/JavaVM.framework/Versions/Current/Commands/java
administrator@test-nearform-macos10 ~iojs % 

[sxa]

So steps required to fix the orka systems:

1. Install a Temurin 17 (The location being used by the nearform makes still contains JDK8 on the orka systems)
2. Ensure that /usr/bin/java points there (or set the path accordingly in the start.sh script
3. Update the agent.jar (was 4.2, needs 4.7)
4. Fix the start.sh so that SECRET is replaced accordingly for the machine (and update PATH, and name of agent.jar if required)

We should check the correct upgrade path should be for java and whether it makes sense to be able to ensure the playbooks can do that.

The above steps have allowed test-orka-macos10.15-x64-1 to come back online, although we will need to ensure that this does not conflict with anything in the playbooks. Note that I've left the old slave.jar on the machine, but it is now using the new agent.jar downloaded from the server.

[richardlau]

@sxa Does the playbook not work?

[sxa]

I haven't tried running - was just looking into the cause :-)
From your comment earlier should the two online 10.14 machines be all "playbooked up" now? -1 and -3 don't seem to have come back yet.

[richardlau]

@sxa I only ran the playbook to update the agent, not the full playbook for Jenkins workers.

[richardlau]

@UlisesGascon is this something you could take a look at?

[UlisesGascon]

Yes, let me see 👍