Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
6e81e11
remove lassen. add matrix
rrsettgast Sep 23, 2025
a637253
Add toss_4_x86_64_ib_cray spack config
victorapm Sep 24, 2025
7d7d85d
Add rocm support to geos spack package
victorapm Sep 24, 2025
482bfb6
Remove old setupLC-TPL script
victorapm Sep 24, 2025
65e46ad
Update TPL scripts with tuolumne
victorapm Sep 24, 2025
f02c85e
Minor fix
victorapm Sep 24, 2025
495ea6e
WIP - Adding ROCm CI
victorapm Sep 24, 2025
ccb12e3
Fixes
victorapm Sep 24, 2025
f50a5dd
Fixes
victorapm Sep 24, 2025
ea8107d
Fixes
victorapm Sep 24, 2025
b7fb42f
Fixes
victorapm Sep 24, 2025
7f27e8a
Fixes
victorapm Sep 24, 2025
a0a92d2
Merge branch 'master' into feature/rocm-ci
victorapm Sep 24, 2025
1f09f8d
Fixes
victorapm Sep 24, 2025
56e2ca8
Fixes
victorapm Sep 24, 2025
dd97285
Fixes
victorapm Sep 24, 2025
1459cd0
Fixes
victorapm Sep 24, 2025
b6e0f74
Fixes
victorapm Sep 24, 2025
5ded729
Fixes
victorapm Sep 24, 2025
696f0b5
Fixes
victorapm Sep 24, 2025
7719d65
Fixes
victorapm Sep 24, 2025
e5d86cd
Update several TPL versions
victorapm Sep 25, 2025
0558fb8
Update spack/uberenv + Add docker/tpl-ubuntu-gcc-hip
victorapm Sep 29, 2025
a687f06
Partial rocm image
victorapm Sep 29, 2025
0451b15
rocsolver
victorapm Sep 29, 2025
c772e7c
rocrand
victorapm Sep 29, 2025
cd3bfaa
Fixes
victorapm Sep 29, 2025
d4045cb
Fixes
victorapm Sep 29, 2025
aa4944f
Fixes
victorapm Sep 29, 2025
a4cff35
Fixes
victorapm Sep 29, 2025
a7e1847
Re-enable jobs
victorapm Sep 29, 2025
9cbb44d
Fixes
victorapm Sep 30, 2025
6735b06
Fixes
victorapm Sep 30, 2025
c0e12b6
Fixes
victorapm Sep 30, 2025
b673869
Fixes
victorapm Sep 30, 2025
e08d41b
Fix
victorapm Sep 30, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions .github/workflows/docker_build_tpls.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,16 +71,22 @@ jobs:
DOCKER_REPOSITORY: geosx/sherlock-gcc10.1.0-openmpi4.1.2-openblas0.3.10-zlib1.2.11
TPL_DOCKERFILE: docker/Stanford/Dockerfile
DOCKER_ROOT_IMAGE: matteofrigo5/sherlock-gcc10.1.0-openmpi4.1.2-cuda12.4.0-openblas0.3.10-zlib1.2.11-cmake3.31.4-no-geosx:0.0.1
SPEC: "%gcc@10.1.0~pygeosx~openmp"
SPEC: "~pygeosx~openmp %c,cxx,fortran=gcc@10.1.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why was this required?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here also, that's the new way spack v1 is handling compilers. See https://github.com/spack/spack/releases#compiler-dependencies

INSTALL_DIR_ROOT: /oak/stanford/groups/tchelepi/geos-sherlock/CPU
RUNS_ON: ubuntu-latest
- name: Sherlock GPU (centos 7.9.2009, gcc 10.1.0, open-mpi 4.1.2, cuda 12.4.0, openblas 0.3.10, zlib 1.2.11)
DOCKER_REPOSITORY: geosx/sherlock-gcc10.1.0-openmpi4.1.2-cuda12.4.0-openblas0.3.10-zlib1.2.11
TPL_DOCKERFILE: docker/Stanford/Dockerfile
DOCKER_ROOT_IMAGE: matteofrigo5/sherlock-gcc10.1.0-openmpi4.1.2-cuda12.4.0-openblas0.3.10-zlib1.2.11-cmake3.31.4-no-geosx:0.0.1
SPEC: "%gcc@10.1.0~pygeosx~openmp cuda_arch=70 ^cuda@12.4.0+allow-unsupported-compilers"
SPEC: "~pygeosx~openmp cuda_arch=70 %c,cxx,fortran=gcc@10.1.0 ^cuda@12.4.0+allow-unsupported-compilers"
INSTALL_DIR_ROOT: /oak/stanford/groups/tchelepi/geos-sherlock/GPU
RUNS_ON: ubuntu-latest
- name: Ubuntu (24.04, gcc 13.3.0 + ROCm 6.4.3)
DOCKER_REPOSITORY: geosx/ubuntu24.04-gcc13.3.0-rocm6.4.3
TPL_DOCKERFILE: docker/tpl-ubuntu-gcc-hip.Dockerfile
DOCKER_ROOT_IMAGE: rocm/dev-ubuntu-24.04:6.4.3
DOCKER_COMPILER_BUILD_ARG: "--build-arg GCC_MAJOR_VERSION=13"
RUNS_ON: ubuntu-latest

steps:
- name: Checkout
Expand Down
2 changes: 1 addition & 1 deletion .uberenv_config.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"package_final_phase": "lvarray_hostconfig",
"package_source_dir": "../..",
"spack_url": "https://github.com/spack/spack",
"spack_commit": "2bfcc69fa870d3c6919be87593f22647981b648a",
"spack_commit": "734c5db2121b01c373eed6538e452f18887e9e44",
"spack_configs_path": "scripts/spack_configs",
"spack_packages_path": "scripts/spack_packages/packages"
}
2 changes: 1 addition & 1 deletion docker/TotalEnergies/Pangea3.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ RUN --mount=src=.,dst=$SRC_DIR,readwrite cd ${SRC_DIR} && \
ln -s /usr/lib64/liblustreapi.so.1 /usr/lib64/liblustreapi.so && \
# Run uberenv
./scripts/uberenv/uberenv.py \
--spec "%gcc@9.4.0+cuda~uncrustify~openmp~pygeosx cuda_arch=70 ^cuda@11.5.0+allow-unsupported-compilers ^caliper~gotcha~sampler~libunwind~libdw~papi" \
--spec "+cuda~uncrustify~openmp~pygeosx cuda_arch=70 %c,cxx,fortran=gcc@9.4.0 ^cuda@11.5.0+allow-unsupported-compilers ^caliper~gotcha~sampler~libunwind~libdw~papi" \
--spack-env-file=${SRC_DIR}/docker/pangea-spack.yaml \
--project-json=.uberenv_config.json \
--prefix ${GEOSX_TPL_DIR} \
Expand Down
152 changes: 152 additions & 0 deletions docker/centos-spack.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
spack:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should not use centos anymore. rocky is the replacement for rhel builds.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good! That simplifies things

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume I can remove the centos build then, otherwise let me know :)

config:
install_tree:
root: $spack/..
projections:
all: '{compiler.name}-{compiler.version}/{name}-{version}-{hash}'
misc_cache: $spack/../misc_cache
test_stage: $spack/../test_stage
build_stage::
- $spack/../build_stage

# Regular TPLs do not need views
view: false

# Include shared variants and versions
include:
- ../defaults.yaml
- ../versions.yaml

packages:
all:
target: [x86_64]
providers:
blas: [netlib-lapack]
lapack: [netlib-lapack]
mpi: [openmpi]

gcc:
version: [8.3.1]
buildable: false
externals:
- spec: gcc@8.3.1 languages:='c,c++,fortran'
prefix: /opt/rh/devtoolset-8/root/usr
extra_attributes:
compilers:
c: /opt/rh/devtoolset-8/root/usr/bin/gcc
cxx: /opt/rh/devtoolset-8/root/usr/bin/g++
fortran: /opt/rh/devtoolset-8/root/usr/bin/gfortran
autoconf:
version: [2.71]
buildable: false
externals:
- spec: autoconf@2.71
prefix: /usr
automake:
version: [1.16.5]
buildable: false
externals:
- spec: automake@1.16.5
prefix: /usr
cmake:
version: [3.28.3]
buildable: false
externals:
- spec: cmake@3.28.3
prefix: /usr/local
cuda:
buildable: False
externals:
- spec: cuda@11.8.0 +allow-unsupported-compilers
prefix: /usr/local/cuda
findutils:
version: [4.7.0]
buildable: false
externals:
- spec: findutils@4.7.0
prefix: /usr
m4:
buildable: false
externals:
- spec: m4@1.4.18
prefix: /usr
mpfr:
buildable: false
externals:
- spec: mpfr@6.0.2
prefix: /usr

# Spack may grab for mpi & we don't want to use them
mpi:
buildable: false

netlib-lapack:
buildable: false
externals:
- spec: netlib-lapack@3.10.0
prefix: /usr
netlib-blas:
buildable: false
externals:
- spec: netlib-blas@3.10.0
prefix: /usr
openmpi:
externals:
- spec: openmpi@4.1.2%clang
prefix: /usr
- spec: openmpi@1.10.7%gcc@8
prefix: /usr/lib64/openmpi
modules: [mpi]
- spec: openmpi@4.1.2%gcc@9
prefix: /usr
- spec: openmpi@4.1.2%gcc@10
prefix: /usr
- spec: openmpi@4.1.2%gcc@11
prefix: /usr
- spec: openmpi@4.1.2%gcc@12
prefix: /usr
perl:
buildable: false
externals:
- spec: perl@5.34.0
prefix: /usr
pkg-config:
buildable: false
externals:
- spec: pkg-config@0.29.2
prefix: /usr
py-sphinx:
buildable: false
externals:
- spec: py-sphinx@4.3.2
prefix: /usr
python:
buildable: false
externals:
- spec: python@3.10.12
prefix: /usr
readline:
buildable: false
externals:
- spec: readline@8.0
prefix: /usr
tar:
buildable: false
externals:
- spec: tar@1.34
prefix: /usr
unzip:
buildable: false
externals:
- spec: unzip@6.0
prefix: /usr
xz:
buildable: false
externals:
- spec: xz@5.2.5
prefix: /usr
zlib:
buildable: false
externals:
- spec: zlib@1.2.11
prefix: /usr
31 changes: 13 additions & 18 deletions docker/pangea-spack.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,32 +17,27 @@ spack:
- ../defaults.yaml
- ../versions.yaml

compilers::
- compiler:
spec: gcc@=9.4.0
paths:
cc: /data_local/sw/spack/0.17.0/opt/spack/linux-rhel8-power9le/gcc-8.4.1/gcc-9.4.0-xe5cqnyajaqz75up3gflln5zlj2rue5v/bin/gcc
cxx: /data_local/sw/spack/0.17.0/opt/spack/linux-rhel8-power9le/gcc-8.4.1/gcc-9.4.0-xe5cqnyajaqz75up3gflln5zlj2rue5v/bin/g++
f77: /data_local/sw/spack/0.17.0/opt/spack/linux-rhel8-power9le/gcc-8.4.1/gcc-9.4.0-xe5cqnyajaqz75up3gflln5zlj2rue5v/bin/gfortran
fc: /data_local/sw/spack/0.17.0/opt/spack/linux-rhel8-power9le/gcc-8.4.1/gcc-9.4.0-xe5cqnyajaqz75up3gflln5zlj2rue5v/bin/gfortran
flags: {}
operating_system: almalinux8
target: ppc64le
modules: []
environment:
prepend_path:
LD_LIBRARY_PATH: /data_local/sw/lsf/10.1/linux3.10-glibc2.17-ppc64le/lib
extra_rpaths: []

packages:
all:
target: [x86_64]
compiler: [gcc]
providers:
blas: [openblas]
lapack: [openblas]
mpi: [openmpi]

gcc:
externals:
- spec: gcc@9.4.0 languages:='c,c++,fortran'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't the default to install all languages?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so, but this was recommended somewhere in the documentation files that I was reading for spack. @bmhan12 could you confirm what is the best thing to do here?

Copy link
Contributor

@bmhan12 bmhan12 Oct 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so, but this was recommended somewhere in the documentation files that I was reading for spack. @bmhan12 could you confirm what is the best thing to do here?

With Spack 1.0, that is the correct way now to specify compilers (compilers are now treated as packages).

However, in addition, if you want to make sure Spack picks your external compiler like we have set before Spack 1.0, you need to add toolchain(s): https://spack.readthedocs.io/en/latest/toolchains_yaml.html

I have a wip branch working through the Spack 1.0/toolchain changes here: https://github.com/GEOS-DEV/thirdPartyLibs/tree/feature/han12/spack_v1_noapi

prefix: /data_local/sw/spack/0.17.0/opt/spack/linux-rhel8-power9le/gcc-8.4.1/gcc-9.4.0-xe5cqnyajaqz75up3gflln5zlj2rue5v
extra_attributes:
compilers:
c: /data_local/sw/spack/0.17.0/opt/spack/linux-rhel8-power9le/gcc-8.4.1/gcc-9.4.0-xe5cqnyajaqz75up3gflln5zlj2rue5v/bin/gcc
cxx: /data_local/sw/spack/0.17.0/opt/spack/linux-rhel8-power9le/gcc-8.4.1/gcc-9.4.0-xe5cqnyajaqz75up3gflln5zlj2rue5v/bin/g++
fortran: /data_local/sw/spack/0.17.0/opt/spack/linux-rhel8-power9le/gcc-8.4.1/gcc-9.4.0-xe5cqnyajaqz75up3gflln5zlj2rue5v/bin/gfortran
environment:
prepend_path:
LD_LIBRARY_PATH: /data_local/sw/lsf/10.1/linux3.10-glibc2.17-ppc64le/lib

autoconf:
externals:
- spec: autoconf@2.69
Expand Down
5 changes: 2 additions & 3 deletions docker/pecan-spack.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ spack:
packages:
all:
target: [x86_64]
compiler: [gcc, clang]
providers:
blas: [intel-mkl]
lapack: [intel-mkl]
Expand Down Expand Up @@ -72,8 +71,8 @@ spack:
buildable: False
externals:
# Pecan CPU/GPU
- spec: intel-mkl@2019.5.281%gcc@8.2.0 threads=openmp
prefix: /opt/intel/compilers_and_libraries_2019.5.281/linux/mkl/
- spec: intel-mkl@2019.5.281 threads=openmp %gcc@8.2.0
prefix: /opt/intel/compilers_and_libraries_2019.5.281/linux/mkl
blas:
buildable: false

Expand Down
53 changes: 20 additions & 33 deletions docker/rocky-spack.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,45 +17,32 @@ spack:
- ../defaults.yaml
- ../versions.yaml

compilers::
- compiler:
extra_rpaths: []
flags:
cflags: -pthread
cxxflags: -pthread
modules: []
operating_system: rocky8
paths:
cc: /usr/bin/clang
cxx: /usr/bin/clang++
f77: /usr/bin/gfortran
fc: /usr/bin/gfortran
spec: clang@17.0.6
target: x86_64
- compiler:
extra_rpaths: []
flags:
cflags: -pthread
cxxflags: -pthread
modules: []
operating_system: rocky8
paths:
cc: /opt/rh/gcc-toolset-13/root/usr/bin/gcc
cxx: /opt/rh/gcc-toolset-13/root/usr/bin/g++
f77: /opt/rh/gcc-toolset-13/root/usr/bin/gfortran
fc: /opt/rh/gcc-toolset-13/root/usr/bin/gfortran
spec: gcc@13.3.1
target: x86_64

packages:
all:
target: [x86_64]
compiler: [gcc, clang]
providers:
blas: [netlib-lapack]
lapack: [netlib-lapack]
mpi: [openmpi]

gcc:
externals:
- spec: gcc@13.3.1 languages:='c,c++,fortran'
prefix: /opt/rh/gcc-toolset-13/root/usr/
extra_attributes:
compilers:
c: /opt/rh/gcc-toolset-13/root/usr/bin/gcc
cxx: /opt/rh/gcc-toolset-13/root/usr/bin/g++
fortran: /opt/rh/gcc-toolset-13/root/usr/bin/gfortran
clang:
externals:
- spec: clang@17 languages:='c,c++'
prefix: /usr
extra_attributes:
compilers:
c: /usr/bin/clang-17
cxx: /usr/bin/clang++-17

Comment on lines +28 to +45
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason for the change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the new way spack v1 is handling compilers. More info here: https://github.com/spack/spack/releases#compiler-dependencies

autoconf:
version: [2.71]
buildable: false
Expand All @@ -77,9 +64,9 @@ spack:
cuda:
buildable: False
externals:
- spec: cuda@12.9.1%gcc@13.3.1 +allow-unsupported-compilers
- spec: cuda@12.9.1 +allow-unsupported-compilers %gcc@13.3.1
prefix: /usr/local/cuda
- spec: cuda@12.9.1%clang@17.0.6 +allow-unsupported-compilers
- spec: cuda@12.9.1 +allow-unsupported-compilers %clang@17.0.6
prefix: /usr/local/cuda
m4:
buildable: false
Expand Down
Loading
Loading