ACCESS-OM2_x_Oceananigans

Trying to "couple" Oceananigans for time-stepping an offline surrogate of ACCESS-OM2 and then use transport matrices to solve for periodic state using a Newon–Krylov solver.

🚧 This is exploratory WIP and may be abandonned any time!

Pipeline

The full pipeline is managed by a unified scripts/driver.sh that submits chained PBS jobs with afterok dependencies. All scripts are model-agnostic — PARENT_MODEL selects the model. Run from the login node:

# Run the full ACCESS-OM2-1 pipeline (default experiment and time window)
PARENT_MODEL=ACCESS-OM2-1 JOB_CHAIN=full bash scripts/driver.sh

# Run the full ACCESS-OM2-025 pipeline
PARENT_MODEL=ACCESS-OM2-025 JOB_CHAIN=full bash scripts/driver.sh

# Specify experiment and time window
PARENT_MODEL=ACCESS-OM2-1 EXPERIMENT=1deg_jra55_ryf9091_gadi TIME_WINDOW=1958-1987 JOB_CHAIN=full bash scripts/driver.sh

Dependency DAG

---
config:
  flowchart:
    curve: basis
    padding: 3
    nodeSpacing: 10
    rankSpacing: 30
---
graph TD
    classDef gpu stroke:#f00;
    subgraph preprocessing
        prep & grid & vel & clo
    end
    subgraph MPIprep
        diagnose_w:::gpu & partition
    end
    subgraph standardruns
        run1yr:::gpu & run1yrfast:::gpu & run10yr:::gpu & run100yr:::gpu & runlong:::gpu
    end
    subgraph TM building
        TMbuild & TMsnapshot
    end
    subgraph solvers
        TMsolve:::gpu & NK:::gpu & run1yrNK:::gpu
    end
    subgraph plotting
        plot1yr & plot10yr & plot100yr & plotTM & plotNK & plotNKtrace & plotMOC
    end
    prep & grid --> vel & clo
    vel --> diagnose_w
    vel & diagnose_w & clo & grid --> partition
    diagnose_w & clo --> run1yr & run1yrfast & run10yr & run100yr & runlong & TMbuild
    partition --> run1yr & run1yrfast & run10yr & run100yr & runlong & TMbuild
    run1yr --> TMsnapshot & plot1yr
    run10yr --> plot10yr
    run100yr --> plot100yr
    TMbuild & TMsnapshot --> TMsolve & NK & plotTM
    NK --> run1yrNK & plotNKtrace
    run1yrNK --> plotNK
    prep & grid --> plotMOC

Loading

Selecting steps with JOB_CHAIN

Use the JOB_CHAIN env var to run only a subset of the pipeline. Steps not in the chain are skipped (their outputs are assumed to already exist). JOB_CHAIN is required — the driver prints usage help if not set.

Steps (topological order): prep grid vel run1yr run10yr run100yr runlong TMbuild TMsnapshot TMsolve NK run1yrNK plotNK plotNKtrace plot1yr plot10yr plot100yr

Shortcuts:

Shortcut	Expands to
preprocessing	prep-grid-vel
standardruns	run1yr-run10yr-run100yr-runlong
TMall	TMbuild-TMsnapshot-TMsolve
plotall	plot1yr-plot10yr-plot100yr-plotNK
full	preprocessing-run1yr-TMall-NK-run1yrNK-plotNK-plot1yr

Range notation: A..B expands to all steps on any path from A to B in the dependency DAG — not a flat list.

All examples below assume PARENT_MODEL=ACCESS-OM2-1; substitute ACCESS-OM2-025 or ACCESS-OM2-01 as needed (PARENT_MODEL is required — there is no default).

# Only run Newton-GMRES solves (matrices must already exist)
PARENT_MODEL=ACCESS-OM2-1 JOB_CHAIN=NK bash scripts/driver.sh

# Run 1-year simulation and plot
PARENT_MODEL=ACCESS-OM2-1 JOB_CHAIN=run1yr-plot1yr bash scripts/driver.sh

# Build matrices and run all solvers
PARENT_MODEL=ACCESS-OM2-1 JOB_CHAIN=run1yr-TMall-NK bash scripts/driver.sh

# Everything from vel to NK (range follows the DAG, excludes run10yr/runlong/TMsolve)
PARENT_MODEL=ACCESS-OM2-1 JOB_CHAIN=vel..NK bash scripts/driver.sh

# Re-run + plot from NK solution (range follows NK→run1yrNK→plotNK path only)
PARENT_MODEL=ACCESS-OM2-1 JOB_CHAIN=run1yrNK..plotNK bash scripts/driver.sh

# Run both const and avg branches
PARENT_MODEL=ACCESS-OM2-1 TM_SOURCE=both JOB_CHAIN=NK-run1yrNK-plotNK bash scripts/driver.sh

# Run preprocessing only
PARENT_MODEL=ACCESS-OM2-1 JOB_CHAIN=preprocessing bash scripts/driver.sh

# Specify experiment and time window
PARENT_MODEL=ACCESS-OM2-1 EXPERIMENT=1deg_jra55_ryf9091_gadi TIME_WINDOW=1958-1987 JOB_CHAIN=full bash scripts/driver.sh

# ACCESS-OM2-025 with specific GPU queue
PARENT_MODEL=ACCESS-OM2-025 GPU_RESOURCES=gpuvolta JOB_CHAIN=run1yr bash scripts/driver.sh

TM_SOURCE filtering

TM_SOURCE controls which transport matrix branch is used for TMsolve, NK, and run1yrNK:

Value	Description
const (default)	Only const-field matrices (from TMbuild)
avg	Only time-averaged snapshot matrices (from TMsnapshot)
both	Both branches in parallel

GPU preprocessing with PREPROCESS_ARCH

Velocity creation can run on GPU for faster processing (grid creation and Python prep always run on CPU):

# Run velocities on GPU
PARENT_MODEL=ACCESS-OM2-1 PREPROCESS_ARCH=GPU JOB_CHAIN=preprocessing bash scripts/driver.sh

Defaults: where they live, what they are

There are two layers:

• Cross-model defaults live in scripts/env_defaults.sh. Sourced by every PBS script and by scripts/driver.sh. Single source of truth.
• Per-model defaults live in model_configs/${PARENT_MODEL}.sh. Sourced first by env_defaults.sh, so per-model values override cross-model fallbacks (and user-set env vars override both).

Julia's side: require_env("VAR") in src/shared_utils/config.jl reads required vars and errors if unset, so a direct julia src/foo.jl invocation without sourcing env_defaults.sh fails fast instead of silently using stale defaults.

The two tables below are generated from those files — regenerate by piping the script output into the README between the marker comments:

bash scripts/print_defaults_tables.sh   # prints the section to stdout

Cross-model defaults (scripts/env_defaults.sh)

Variable	Default	Notes
TIME_WINDOW	1968-1977
W_FORMULATION	wprescribed	wdiagnosed | wprescribed
PRESCRIBED_W_SOURCE	parent	diagnosed | parent (only when W_FORMULATION=wprescribed)
ADVECTION_SCHEME	centered2	centered2 | weno3 | weno5
TIMESTEPPER	AB2	AB2 | SRK2 | SRK3 | SRK4 | SRK5
PLOT_TS	no	yes | no — opt-in T/S surface animations in plot_standardrun_age.jl
TRACE_SOLVER_HISTORY	yes	yes | no — save Newton iterates xₙ as newton_iterate_NN.jld2 (use INITIAL_AGE=latest to restart)
JVP_METHOD	exact	exact | fd — Jacobian-vector product method for NK
LINEAR_SOLVER	Pardiso	Pardiso | ParU | UMFPACK
MATRIX_PROCESSING	symdrop	raw | symfill | dropzeros | symdrop
INITIAL_AGE	0	0 | TMage | latest | <path to .jld2>
TM_SOURCE	const	const | avg
TM_MODEL_CONFIG	(empty)	override MODEL_CONFIG used to locate NK's preconditioner TM (empty = use MODEL_CONFIG)
GM_REDI	no	no | diff | adv (legacy: yes = diff)
MONTHLY_KAPPAV	yes	yes | no — derive 3D κV on the fly from 2D monthly MLD (tags MODEL_CONFIG with _mkappaV); default yes
IMPLICIT_KAPPAV	yes	yes | no — when "no", drop implicit vertical-diffusion closure (Probe B); tags MODEL_CONFIG with _noKV
TBLOCKING	no	no | integer K ≥ 2 (temporal blocking: K sub-steps per MPI exchange)
GRID_HX	7	grid halo in x (≥ K+1 when TBLOCKING=K)
GRID_HY	7	grid halo in y (≥ K+1 when TBLOCKING=K)
GRID_HZ	2	grid halo in z (2 sufficient; larger is harmless)
LOAD_BALANCE	surface	no | surface | cell | mix | minmax | yes(=surface; back-compat) — only valid when PARTITION_X=1. Auto-suppressed in MODEL_CONFIG when RANKS=1 (serial).
ACTIVE_CELLS_MAP	yes	yes | no — when "no", build IBG with active_cells_map=false and tag output files with _noACM
TRAF	no	yes | no — Time-Reversed Adjoint Flow (adjoint age via reversed monthly FTS + sign-flipped u, v)
TRAF_TM_SOURCE	invVMtV	invVMtV | M_traf — matrix to use for TMsolve/NK when TRAF=yes (ignored when TRAF=no)
OMEGA	all	all | z — restrict the age source to where z_center < - m (filename suffix only)
MPI_BINDING	numa	numa | socket | none — mpiexec --bind-to / --map-by binding policy
CPU_QUEUE	express
CHECK_BOUNDS	no

Per-model defaults (model_configs/)

Variable	OM2-1	OM2-025	OM2-01
GPU_QUEUE	gpuvolta	gpuhopper	gpuhopper
PARTITION	1x1	1x2	1x4
VELOCITY_SOURCE	totaltransport	totaltransport	cgridtransports
TIMESTEP_MULT	4	3	2
LUMP_AND_SPRAY	2x2	2x2	5x5
KAPPA_H	300	75	30
KAPPA_V_ML	1e-1	5e-2	25e-3
KAPPA_V_BG	3e-5	15e-6	75e-7
CPU_QUEUE	—	hugemem	megamem

Resource knobs (walltimes, memory limits, queue choices for individual PBS jobs) also live in model_configs/*.sh but are omitted from this table — see the files for the full list.

Model notes

• OM2-01 (0.1°) has no GM transport. The model is eddy-resolving, so the Gent–McWilliams parameterization is disabled. There is no ty_trans_rho_gm (or tx_trans_rho_gm, ty_trans_gm, etc.) in the intake catalog for any OM2-01 experiment. Scripts that read GM variables must treat them as optional (zero them out when missing) for OM2-01.

Script organisation

scripts/
├── driver.sh                  # Unified pipeline entry point
├── test_driver.sh             # Test/diagnostic driver (halofill, diag, mpi)
├── env_defaults.sh            # Common env var defaults
├── prepreprocessing/          # Python preprocessing (periodicaverage.py PBS wrapper)
├── preprocessing/             # Grid, velocities, transport matrices
├── standard_runs/             # Age simulations (1yr, 10yr, 100yr, long, benchmark)
├── solvers/                   # Newton-Krylov + TM age solvers
├── plotting/                  # Diagnostic plots + architecture comparison
├── tests/                     # Test PBS wrappers (halofill, diag, mpi)
├── benchmarks/                # Parameter sweep submitters
├── maintenance/               # Package management, MPI setup, archiving
└── debugging/                 # Debug/check scripts

Tracking submissions

Every scripts/driver.sh invocation drops a TOML manifest next to its outputs and appends one row per submitted PBS job to a global TSV index. Together they answer "what did I submit, when, how, and how did it turn out?" without grep'ing PBS history.

Per-case launcher (optional)

For one-off comparison runs, define a case file under scripts/runs/cases/{name}.sh that just exports env vars (mirroring the model_configs/{PARENT_MODEL}.sh pattern) and submit it with the launcher:

bash scripts/runs/run_case.sh scripts/runs/cases/OM2-1_TR1968-1977_MLD1972.sh

The launcher sources the case file (so its exports become env vars), sets CASE_FILE so the manifest records which case was used, and execs scripts/driver.sh.

Per-run manifest

Dropped at:

outputs/{PM}/{EXP}/{output_tag}/manifests/{timestamp}_{pid}.toml

Contents: [meta] (timestamp, user, host, case_file), [git] (commit, branch, dirty), [env] (every key in COMMON_VARS plus JOB_CHAIN, TM_SOURCE, etc.), and a [[jobs]] table with one entry per PBS job (step, jobid, script, deps).

Submissions index

scripts/runs/submissions.tsv — append-only, one row per PBS job, 20 columns:

timestamp jobid step deps manifest_path case_file git_commit
JOB_CHAIN PARENT_MODEL TIME_WINDOW MLD_TIME_WINDOW script
exit_code queue walltime_req walltime_used mem_req mem_used ncpus ngpus

The first 12 columns are written at submit time; the 8 trailing PBS-side columns are filled later by reconcile.

The TSV is .gitignored (it's mutable runtime state); the manifests are the source of truth, the TSV is just a finding aid.

When to run reconcile

Run after a batch of jobs has finished (or whenever you want up-to-date PBS state in the TSV):

bash scripts/runs/reconcile_submissions.sh

The script queries qstat -fx <jobid> for each row missing PBS-side fields and rewrites the TSV in place. It's idempotent and safe to run repeatedly. Behaviour per row:

State	Action
DRY_RUN_*	exit_code=DRY, others -
Still Q/H/R	leave PBS fields empty (next reconcile picks up)
Finished, in qstat history	fill all 8 fields; mem normalised to GB
Aged out of qstat	exit_code=?, others ? (preserves prior values if any)

PBS history retention on Gadi is ~7 days, so reconcile within that window or the row sticks at ?.

Tip: reconcile near job completion to avoid losing data; if you wait weeks, the values won't be recoverable.

Rebuilding the TSV from manifests

If the index is lost, corrupted, or schema-drifted, rebuild it from the manifests (the source of truth):

python3 scripts/runs/rebuild_submissions.py
bash scripts/runs/reconcile_submissions.sh

rebuild_submissions.py walks outputs/**/manifests/*.toml and emits one TSV row per [[jobs]] entry with the 12 stable fields populated. Reconcile then fills the PBS-side fields.

Multi-GPU (MPI) runs

Multi-GPU simulations use MPI to distribute the grid across GPUs. All PBS scripts automatically detect NGPUS > 1 and launch via mpiexec.

Socket binding on Gadi: Gadi assigns MPI ranks to CPU sockets randomly by default. Since each GPU is physically attached to a specific CPU socket, this can result in a CPU communicating with a GPU on a different socket, causing severe CPU-GPU transfer slowdowns. All scripts use --bind-to socket --map-by socket to pin each MPI rank to the socket directly connected to its GPU.

GPU partition is set via GPU_RESOURCES:

# 2x2 partition (4 GPUs) on Volta
PARENT_MODEL=ACCESS-OM2-1 GPU_RESOURCES=gpuvolta-2x2 JOB_CHAIN=run1yr bash scripts/driver.sh

# 1x2 slab partition (2 GPUs) on Hopper
PARENT_MODEL=ACCESS-OM2-025 GPU_RESOURCES=gpuhopper-1x2 JOB_CHAIN=run1yr bash scripts/driver.sh

GitHub CLI (gh)

To use the gh CLI on Gadi, load the module first:

module use /g/data/vk83/modules
module load system-tools/gh

Project setup notes

Gadi compute nodes don't have access to the internet, so the project dependencies must be downloaded on the login node. But the default multi-threaded precompilation could use too much resources and crash during pkg> up. Instead, run the dedicated script scripts/maintenance/pkg_update_project.sh, which runs pkg> up on the login node without precompilation, then submits precompilation on compute nodes on the CPU and then on the GPU.

Configuration

Simulations are configured via environment variables.

Experiment and time window

Variable	Default	Description
EXPERIMENT	1deg_jra55_iaf_omip2_cycle6 (OM2-1) or 025deg_jra55_iaf_omip2_cycle6 (OM2-025)	Intake catalog key for ACCESS-OM2 experiment
TIME_WINDOW	1968-1977	Year range YYYY-YYYY or single year YYYY

These determine the input data source and the directory structure for preprocessed inputs, outputs, and logs.

Model config

The 4 core config variables determine the model setup and output directory paths:

Variable	Valid values	Default	Description
VELOCITY_SOURCE	cgridtransports, totaltransport	per-model: totaltransport for OM2-1/025, cgridtransports for OM2-01	Source of prescribed velocities (totaltransport = resolved + GM from parent)
W_FORMULATION	wdiagnosed, wprescribed	wprescribed	Vertical velocity treatment
ADVECTION_SCHEME	centered2, weno3, weno5	centered2	Tracer advection scheme
TIMESTEPPER	AB2, SRK2, SRK3, SRK4, SRK5	AB2	Time-stepping scheme
GM_REDI	no, diff, adv	no	Online Redi-GM parameterization (diffusive or advective formulation)
KAPPA_H	positive float (m²/s)	per-model: 300/75/30	Horizontal scalar diffusivity; also used for the GM-Redi isopycnal κ
KAPPA_V_ML	positive float (m²/s)	per-model: 1e-1/5e-2/25e-3	Vertical diffusivity in the mixed layer
KAPPA_V_BG	positive float (m²/s)	per-model: 3e-5/15e-6/75e-7	Vertical (background) diffusivity in the ocean interior

Timestepper values map to Oceananigans symbols:

• AB2 = :QuasiAdamsBashforth2 (default quasi-Adams-Bashforth 2nd order)
• SRK{N} = :SplitRungeKutta{N} (split Runge-Kutta with N = 2..5 stages)

The combined tag MODEL_CONFIG = {VS}_{WF}_{AS}_{TS} (e.g. totaltransport_wparent_centered2_AB2 with default OM2-1 settings) determines output directory paths and log filenames. The diffusivity tags _kH{KAPPA_H}_kVML{KAPPA_V_ML}_kVBG{KAPPA_V_BG} are always appended (e.g. _kH300_kVML1e-1_kVBG3e-5 for OM2-1, _kH75_kVML5e-2_kVBG15e-6 for OM2-025) — the env-var string forms are embedded verbatim, so they must parse to a float and stay free of . to keep paths clean. Further suffixes are appended for non-default flags: _GMREDI/_GMREDIadv (Redi-GM), _mkappaV (MONTHLY_KAPPAV=yes), _noKV (IMPLICIT_KAPPAV=no), _TB<K> (temporal blocking), _LB{S,…} (load-balanced partition), _DTx<M> (TIMESTEP_MULT>1), _traf (TRAF), _noACM.

Resolution scaling of the diffusivities

The KAPPA_* defaults shrink as the grid refines, set per model in model_configs/*.sh:

Model	KAPPA_H	KAPPA_V_ML	KAPPA_V_BG
OM2-1	300	1e-1	3e-5
OM2-025	75	5e-2	15e-6
OM2-01	30	25e-3	75e-7

• κH scales by the ratio of √(cell area): OM2-025 has Δx,Δy ÷4 ⇒ κH ÷4 (300→75); OM2-01 has Δx ÷10 ⇒ κH ÷10 (300→30). The GM-Redi isopycnal κ is tied to the same value.
• κV scales by √(level-ratio × Δx-ratio): OM2-025 = √(1×4) = 2 (κV ÷2); OM2-01 = √(1.5×10) = √15 ≈ 3.87. OM2-01 uses rounded values (≈÷4: 0.025, 7.5e-6) directly rather than the exact √15 factor, for cleaner tags.

GM transport options

There are four ways to include GM (Gent-McWilliams) effects in the tracer transport. See docs/GM_TRANSPORT.md for full details.

# Parent-model GM by default for OM2-1 / OM2-025 (resolved + GM combined velocities)
PARENT_MODEL=ACCESS-OM2-1 JOB_CHAIN=full bash scripts/driver.sh

# Resolved-only velocities (no GM)
PARENT_MODEL=ACCESS-OM2-1 VELOCITY_SOURCE=cgridtransports JOB_CHAIN=full bash scripts/driver.sh

# Online diffusive Redi-GM
PARENT_MODEL=ACCESS-OM2-1 GM_REDI=diff JOB_CHAIN=full bash scripts/driver.sh

# Online advective Redi-GM
PARENT_MODEL=ACCESS-OM2-1 GM_REDI=adv JOB_CHAIN=full bash scripts/driver.sh

Note: OM2-01 defaults to cgridtransports (no separate mass-transport intake) and so has no GM-by-default option; for that model GM must come via GM_REDI=diff or GM_REDI=adv.

Solver-specific variables

These configure the fixed-point acceleration solvers in solve_periodic_AA.jl (archived):

Variable	Default	Description
AA_M	40	Anderson history size (used by NLsolve, SIAMFANL, FixedPoint)
NLSAA_BETA	1.0	Anderson damping parameter (try 0.5 for slow convergence)
SMAA_SIGMA_MIN	0.0	SpeedMapping minimum σ; setting to 1 may avoid stalling
SMAA_STABILIZE	no	Stabilization mapping before extrapolation (yes/no)
SMAA_CHECK_OBJ	no	Restart at best past iterate on NaN/Inf (yes/no)
SMAA_ORDERS	332	Alternating order sequence (each digit 1–3)

Shell defaults are set in scripts/env_defaults.sh, which is sourced by all PBS job scripts. Override at submission time:

qsub -v TIMESTEPPER=SRK3,ADVECTION_SCHEME=weno5 scripts/standard_runs/run_1year.sh

Tests

Test/diagnostic jobs are managed by a separate scripts/test_driver.sh (independent from the production driver.sh). Available test steps:

Step	Description
halofill	MWE testing fill_halo_regions! at all staggered locations on distributed tripolar grids
diag	10-step diagnostic run saving age at every time step (for serial vs distributed comparison)
mpi	MPI smoke test (rank/device info, 10-iteration simulation)

# Run halo fill test on 4 GPUs (2x2 partition)
GPU_RESOURCES=gpuvolta-2x2 PARENT_MODEL=ACCESS-OM2-1 JOB_CHAIN=halofill bash scripts/test_driver.sh

# Run diagnostic steps (serial baseline)
PARENT_MODEL=ACCESS-OM2-1 JOB_CHAIN=diag bash scripts/test_driver.sh

# Run diagnostic steps (distributed 2x2)
GPU_RESOURCES=gpuvolta-2x2 PARENT_MODEL=ACCESS-OM2-1 JOB_CHAIN=diag bash scripts/test_driver.sh

# Run all tests at once
GPU_RESOURCES=gpuvolta-2x2 PARENT_MODEL=ACCESS-OM2-1 JOB_CHAIN=halofill-diag-mpi bash scripts/test_driver.sh

Comparing serial vs distributed output

After both serial and distributed diag jobs complete, compare step-by-step:

GPU_TAG=2x2 DURATION_TAG=diag PARENT_MODEL=ACCESS-OM2-1 \
  qsub scripts/plotting/compare_runs_across_architectures.sh

This prints a per-step volume-weighted RMS norm table and generates diagnostic plots. The same script works for 1-year runs (DURATION_TAG=1year).

Matrix regression tests

Julia test scripts for matrix regression live in test/. To run the regression test comparing newly-built snapshot matrices against archived reference matrices:

qsub scripts/debugging/check_snapshot_matrices_job.sh

Preprocessed outputs layout

Preprocessing writes data and images under:

preprocessed_inputs/<PARENT_MODEL>/<EXPERIMENT>/

Grid file (shared across time windows):

• grid.jld2

Per-time-window data files under <TIME_WINDOW>/monthly/ and <TIME_WINDOW>/yearly/:

• u_interpolated_monthly.jld2 / u_interpolated_yearly.jld2
• v_interpolated_monthly.jld2 / v_interpolated_yearly.jld2
• w_monthly.jld2 / w_yearly.jld2
• eta_monthly.jld2 / eta_yearly.jld2
• u_from_mass_transport_monthly.jld2 / u_from_mass_transport_yearly.jld2
• v_from_mass_transport_monthly.jld2 / v_from_mass_transport_yearly.jld2
• w_from_mass_transport_monthly.jld2 / w_from_mass_transport_yearly.jld2
• u_from_total_transport_monthly.jld2 / u_from_total_transport_yearly.jld2 (if GM data available)
• v_from_total_transport_monthly.jld2 / v_from_total_transport_yearly.jld2 (if GM data available)
• w_from_total_transport_monthly.jld2 / w_from_total_transport_yearly.jld2 (if GM data available)

NetCDF climatologies from periodicaverage.py:

• <TIME_WINDOW>/monthly/*_monthly.nc
• <TIME_WINDOW>/yearly/*_yearly.nc

Plots are colocated under:

preprocessed_inputs/<PARENT_MODEL>/<EXPERIMENT>/<TIME_WINDOW>/monthly/plots/

with subdirectories for each plotted field family:

• u/ (original B-grid u)
• v/ (original B-grid v)
• u_interpolated/
• v_interpolated/
• w/
• eta/
• u_from_mass_transport/
• v_from_mass_transport/
• w_from_mass_transport/

Each plot subdirectory is split by vertical level (k<level> when applicable). Each image contains one field only.