Skip to content

Conversation

@arporter
Copy link
Member

@arporter arporter commented Oct 23, 2025

This update is needed so we can update performance results for The Paper.

@arporter arporter marked this pull request as draft October 23, 2025 11:18
@arporter arporter self-assigned this Oct 23, 2025
@arporter arporter changed the title Update to latest PSyclone Update tracer-advection benchmarks to latest PSyclone Oct 23, 2025
@arporter
Copy link
Member Author

arporter commented Oct 23, 2025

I had to update the version of Ubuntu used in the GHA runner and unfortunately it seems that more recent versions of gcc can't build the version of Kokkos that we have in the submodule. We get, e.g.:

../../../../../shared/kokkos/core/src/impl/Kokkos_MemoryPool.cpp:112:48: error: ‘uint32_t’ has not been declared
  112 | void _print_memory_pool_state(std::ostream& s, uint32_t const* sb_state_ptr,
      |                                                ^~~~~~~~
../../../../../shared/kokkos/core/src/impl/Kokkos_MemoryPool.cpp:113:49: error: ‘uint32_t’ has not been declared

which the compiler helpfully says can possibly be fixed:

../../../../../shared/kokkos/core/src/impl/Kokkos_MemoryPool.cpp:49:1: note: ‘uint32_t’ is defined in header ‘<cstdint>; this is probably fixable by adding ‘#include <cstdint>’

However, we should probably update the kokkos submodule. I tried doing that but then the (generated?) Makefile files were missing from the kokkos directory so I defer to @sergisiso on that (see the last test failure).

@arporter arporter added the question Further information is requested label Oct 23, 2025
@arporter arporter changed the title Update tracer-advection benchmarks to latest PSyclone Update to PSyclone 3.2.x Oct 23, 2025
@arporter
Copy link
Member Author

Ready for review/assistance with Kokkos frm @sergisiso or @LonelyCat124

@arporter
Copy link
Member Author

I've not made any attempt to improve the optimisations that the various scripts perform - I've just modernised them.

@arporter arporter marked this pull request as ready for review October 23, 2025 15:36
@LonelyCat124
Copy link
Collaborator

LonelyCat124 commented Oct 27, 2025

One thing I noticed is that this repo is pinned to a very old version of Kokkos (ae5fc649e). If you switch to develop these errors go away, but I'm getting a link created to a Makefile.kokkos.f90 file, which doesn't exist, and I assume is just a missing file, but I can't see what creates it? Without this in the makefile it doesn't work either, a lot of failure to find modules (I didn't have kokkos on my paths at this stage, but the README doesn't expect that). This is just repeating what Andy said, but I am concerned that however we have setup using Kokkos might need an entire redo.

All the PSyclone scripts for NEMOLite2D worked correctly for me, and everything in tracer advection seemed ok. I didn't have openacc or opencl to test though.

@LonelyCat124
Copy link
Collaborator

LonelyCat124 commented Oct 28, 2025

On new machine (with nvfortran 25.5) I get this warning:
nvfortran-Warning- The -gpu=[no]managed option is deprecated; please use -gpu=mem:managed or -gpu=mem:separate instead

Edit: Also the taskset in problemsize.sh setting affinity to 2 seems arbitrary, do we know why its set like that?

Edit2: Another error when trying acc nvfortran-Error-The flag -Mcuda is no longer supported, please use -cuda and -gpu instead.

Edit3: References to -lnvToolsExt should be replaced by -lnvtx3interop to work with newer CUDA.

Edit4: xiar is gone from intel compiler suite as of 2025.1 ( i swapped to ar and it worked fine on coombs). -qopt-report max level is now 3.

@sergisiso
Copy link
Collaborator

@arporter The makefile was deprecated in the recently released kokkos 5, I suggest we stick with kokkos tag 4.7.00 unless we want to change to cmake or use a preinstalled kokkos instead of embeding it.

@LonelyCat124 I agree the nvfortran flags need updating, also we can get rid of the taskset, it was probably very specific to the last time we evaluated it.

Copy link
Collaborator

@sergisiso sergisiso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arporter I pushed a small commit that fixes the kokkos situation, see below other comments

@sergisiso sergisiso added reviewed with actions and removed question Further information is requested labels Dec 9, 2025
@arporter
Copy link
Member Author

Thanks both. I've updated the compiler flags and responded to Sergi's comments. Ready for another look now.

Copy link
Collaborator

@sergisiso sergisiso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All comments have been addressed and all CI passes. Thanks for updating this @arporter.

@sergisiso sergisiso merged commit 4d4b264 into master Dec 19, 2025
1 check passed
@sergisiso
Copy link
Collaborator

I won't delete the branch just yet as @LonelyCat124 has referenced it a recent email to AMD and I don't want to cause any confusion, but we can delete this after some time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants