Skip to content

Add support in convert.nemo_to_sgrid for depthw coordinate#2626

Merged
VeckoTheGecko merged 6 commits intoParcels-code:mainfrom
VeckoTheGecko:push-rttuqnslyyqu
May 6, 2026
Merged

Add support in convert.nemo_to_sgrid for depthw coordinate#2626
VeckoTheGecko merged 6 commits intoParcels-code:mainfrom
VeckoTheGecko:push-rttuqnslyyqu

Conversation

@VeckoTheGecko
Copy link
Copy Markdown
Contributor

Description

Related to getting the benchmark suite running (including the prior PR - #2605 ) this PR does the following:

  • Adds a test test_convert.py::test_nemo_to_sgrid_with_depth
  • Adds a metadata-only version of the original source dataset in the benchmark repo (for use in test_nemo_to_sgrid_with_depth)
  • Updates the convert module to allow for the depthw dimension to be optionally provided
  • Adds type hinting to the convert module allowing for easier refactoring/overview

Still need to double check that this fixes all the problems in the benchmark repo - will do that on Wednesday. If you could review the code in the meantime that would be great @erikvansebille :)

Checklist

  • Closes None
  • Tests added
  • This PR targets the correct branch (main for normal development, v3-support for v3 support)

AI Disclosure

None used

Comment thread src/parcels/convert.py
_NEMO_EXPECTED_COORDS = [
"glamf",
"gphif",
] # "depthw" # TODO: Depthw needs to be available if the data has a depth dim. Refactor the whole convert module, this can surely all be handled better.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Refactor the whole convert module, this can surely all be handled better.

Is still on my TODO list (though quite low down). I have ideas on how we can make the conversion very robust but I want that to come later.

Also make depthw an optional coord for NEMO
@VeckoTheGecko
Copy link
Copy Markdown
Contributor Author

Still need to double check that this fixes all the problems in the benchmark repo - will do that on Wednesday

I just did, and I think its working - though didn't have the time to wait for the run to complete. I think that's the last of the bughunting - working on perf on Wed

Copy link
Copy Markdown
Member

@erikvansebille erikvansebille left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Just one small question below. So the benchmarks run with this PR?

Comment thread tests/test_convert.py Outdated
@VeckoTheGecko
Copy link
Copy Markdown
Contributor Author

So the benchmarks run with this PR?

Just spent today getting Lorenz setup with a workspace to run the benchmarks.

Unfortunately the benchmarks don't pass - getting the following error:

/nethome/hodgs004/.pixi/bin/pixi
/storage/shared/oceanparcels/output_data/data_nick/benchmarking-data-20260506
Copied catalogue across to /storage/shared/oceanparcels/output_data/data_nick/benchmarking-data-20260506/surf-data/parcels-examples/catalog.yml
Output directory already exists! Exiting...
Copied catalogue across to /storage/shared/oceanparcels/output_data/data_nick/benchmarking-data-20260506/surf-data/parcels-benchmarks/catalog.yml
Output directory already exists! Exiting...
· Discovering benchmarks
· Running 6 total benchmarks (1 commits * 1 environments * 6 benchmarks)
[ 0.00%] ·· Benchmarking existing-py_storage_home_hodgs004_projects_benchmarking_parcels-benchmarks_.pixi_envs_default_bin_python
[16.67%] ··· Running (fesom2.FESOM2.time_load_data--)....
[58.33%] ··· fesom2.FESOM2.peakmem_pset_execute                          failed
[58.33%] ··· ======= ============================
             --               integrator         
             ------- ----------------------------
              npart   <function AdvectionRK2_3D> 
             ======= ============================
              10000             failed           
             ======= ============================
             For parameters: 10000, <function AdvectionRK2_3D>
             INFO: Using known vertical dimension mapping: 'nz' (interfaces) and 'nz1' (centers).
             INFO: Renaming vertical dimensions: {'nz': 'zf', 'nz1': 'zc'}
             INFO: cf_xarray found variable 'w' with CF standard name 'w' in dataset, renamed it to 'W' for Parcels simulation.
             INFO: cf_xarray found variable 'unod' with CF standard name 'unod' in dataset, renamed it to 'U' for Parcels simulation.
             INFO: cf_xarray found variable 'vnod' with CF standard name 'vnod' in dataset, renamed it to 'V' for Parcels simulation.
             
               0%|          | 0/86400.0 [00:00<?, ?it/s]
             Integration time: 0.0:   0%|          | 0/86400.0 [00:00<?, ?it/s]Traceback (most recent call last):
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/server.py", line 179, in _run_server
                 _run(run_args)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/run.py", line 72, in _run
                 result = benchmark.do_run()
                          ^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/benchmarks/_base.py", line 661, in do_run
                 return self.run(*self._current_params)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/benchmarks/peakmem.py", line 66, in run
                 self.func(*param)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/benchmarks/fesom2.py", line 53, in peakmem_pset_execute
                 self.pset_execute(npart, integrator)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/benchmarks/fesom2.py", line 47, in pset_execute
                 pset.execute(kernels=integrator, runtime=runtime, dt=dt)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/_core/particleset.py", line 438, in execute
                 self._kernel.execute(self, endtime=next_time, dt=dt)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/_core/kernel.py", line 209, in execute
                 f(pset[evaluate_particles], self._fieldset)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/kernels/_advection.py", line 47, in AdvectionRK2_3D
                 (u1, v1, w1) = fieldset.UVW[particles]
                                ~~~~~~~~~~~~^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/_core/field.py", line 337, in __getitem__
                 return self.eval(key.time, key.z, key.lat, key.lon, key)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/_core/field.py", line 324, in eval
                 (u, v, w) = self._vector_interp_method(particle_positions, grid_positions, self)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/interpolators/_uxinterpolators.py", line 102, in Ux_Velocity
                 u = vectorfield.U._interp_method(particle_positions, grid_positions, vectorfield.U)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/interpolators/_uxinterpolators.py", line 20, in UxConstantFaceConstantZC
                 return field.data.values[
                        ^^^^^^^^^^^^^^^^^^
             IndexError: index 60403 is out of bounds for axis 2 with size 60277
             
             Integration time: 0.0:   0%|          | 0/86400.0 [00:01<?, ?it/s]

[66.67%] ··· fesom2.FESOM2.time_load_data                                    ok
[66.67%] ··· ======= ============================
             --               integrator         
             ------- ----------------------------
              npart   <function AdvectionRK2_3D> 
             ======= ============================
              10000            234±2ms           
             ======= ============================

[75.00%] ··· fesom2.FESOM2.time_pset_execute                             failed
[75.00%] ··· ======= ============================
             --               integrator         
             ------- ----------------------------
              npart   <function AdvectionRK2_3D> 
             ======= ============================
              10000             failed           
             ======= ============================
             For parameters: 10000, <function AdvectionRK2_3D>
             INFO: Using known vertical dimension mapping: 'nz' (interfaces) and 'nz1' (centers).
             INFO: Renaming vertical dimensions: {'nz': 'zf', 'nz1': 'zc'}
             INFO: cf_xarray found variable 'w' with CF standard name 'w' in dataset, renamed it to 'W' for Parcels simulation.
             INFO: cf_xarray found variable 'unod' with CF standard name 'unod' in dataset, renamed it to 'U' for Parcels simulation.
             INFO: cf_xarray found variable 'vnod' with CF standard name 'vnod' in dataset, renamed it to 'V' for Parcels simulation.
             
               0%|          | 0/86400.0 [00:00<?, ?it/s]
             Integration time: 0.0:   0%|          | 0/86400.0 [00:00<?, ?it/s]Traceback (most recent call last):
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/server.py", line 179, in _run_server
                 _run(run_args)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/run.py", line 72, in _run
                 result = benchmark.do_run()
                          ^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/benchmarks/_base.py", line 661, in do_run
                 return self.run(*self._current_params)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/benchmarks/time.py", line 165, in run
                 samples, number = self.benchmark_timing(
                                   ^^^^^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/benchmarks/time.py", line 258, in benchmark_timing
                 timing = timer.timeit(number)
                          ^^^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/timeit.py", line 180, in timeit
                 timing = self.inner(it, self.timer)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^
               File "<timeit-src>", line 6, in inner
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/benchmarks/time.py", line 90, in func
                 self.func(*param)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/benchmarks/fesom2.py", line 50, in time_pset_execute
                 self.pset_execute(npart, integrator)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/benchmarks/fesom2.py", line 47, in pset_execute
                 pset.execute(kernels=integrator, runtime=runtime, dt=dt)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/_core/particleset.py", line 438, in execute
                 self._kernel.execute(self, endtime=next_time, dt=dt)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/_core/kernel.py", line 209, in execute
                 f(pset[evaluate_particles], self._fieldset)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/kernels/_advection.py", line 47, in AdvectionRK2_3D
                 (u1, v1, w1) = fieldset.UVW[particles]
                                ~~~~~~~~~~~~^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/_core/field.py", line 337, in __getitem__
                 return self.eval(key.time, key.z, key.lat, key.lon, key)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/_core/field.py", line 324, in eval
                 (u, v, w) = self._vector_interp_method(particle_positions, grid_positions, self)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/interpolators/_uxinterpolators.py", line 102, in Ux_Velocity
                 u = vectorfield.U._interp_method(particle_positions, grid_positions, vectorfield.U)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/interpolators/_uxinterpolators.py", line 20, in UxConstantFaceConstantZC
                 return field.data.values[
                        ^^^^^^^^^^^^^^^^^^
             IndexError: index 60403 is out of bounds for axis 2 with size 60277
             
             Integration time: 0.0:   0%|          | 0/86400.0 [00:01<?, ?it/s]

[83.33%] ··· ...inear.MOICurvilinear.peakmem_pset_execute_3d             failed
[83.33%] ··· ============== =============
             --             chunk / npart
             -------------- -------------
              interpolator   256 / 10000 
             ============== =============
                XLinear         failed   
             ============== =============
             For parameters: 'XLinear', 256, 10000
             Traceback (most recent call last):
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/server.py", line 179, in _run_server
                 _run(run_args)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/run.py", line 72, in _run
                 result = benchmark.do_run()
                          ^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/benchmarks/_base.py", line 661, in do_run
                 return self.run(*self._current_params)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/benchmarks/peakmem.py", line 66, in run
                 self.func(*param)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/benchmarks/moi_curvilinear.py", line 79, in peakmem_pset_execute_3d
                 self.pset_execute_3d(interpolator, chunk, npart)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/benchmarks/moi_curvilinear.py", line 71, in pset_execute_3d
                 pset.execute(
               File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/_core/particleset.py", line 438, in execute
                 self._kernel.execute(self, endtime=next_time, dt=dt)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/_core/kernel.py", line 244, in execute
                 error_func(pset[inds].z, pset[inds].lat, pset[inds].lon)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/_core/statuscodes.py", line 85, in _raise_grid_searching_error
                 raise GridSearchingError(f"Grid searching failed at (z={z}, lat={y}, lon={x})")
             parcels._core.statuscodes.GridSearchingError: Grid searching failed at (z=array([0.], dtype=float32), lat=array([-21.261051], dtype=float32), lon=array([7.4583335], dtype=float32))

[91.67%] ··· ...curvilinear.MOICurvilinear.time_load_data_3d             failed
[91.67%] ··· ============== =============
             --             chunk / npart
             -------------- -------------
              interpolator   256 / 10000 
             ============== =============
                XLinear         failed   
             ============== =============
             For parameters: 'XLinear', 256, 10000
             Traceback (most recent call last):
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/xarray/core/dataset.py", line 1237, in _construct_dataarray
                 variable = self._variables[name]
                            ~~~~~~~~~~~~~~~^^^^^^
             KeyError: 'deptht'
             
             During handling of the above exception, another exception occurred:
             
             Traceback (most recent call last):
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/xarray/core/dataset.py", line 1344, in __getitem__
                 return self._construct_dataarray(key)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/xarray/core/dataset.py", line 1239, in _construct_dataarray
                 _, name, variable = _get_virtual_variable(self._variables, name, self.sizes)
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/xarray/core/dataset_utils.py", line 79, in _get_virtual_variable
                 raise KeyError(key)
             KeyError: 'deptht'
             
             The above exception was the direct cause of the following exception:
             
             Traceback (most recent call last):
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/server.py", line 179, in _run_server
                 _run(run_args)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/run.py", line 72, in _run
                 result = benchmark.do_run()
                          ^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/benchmarks/_base.py", line 661, in do_run
                 return self.run(*self._current_params)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/benchmarks/time.py", line 165, in run
                 samples, number = self.benchmark_timing(
                                   ^^^^^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/benchmarks/time.py", line 258, in benchmark_timing
                 timing = timer.timeit(number)
                          ^^^^^^^^^^^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/timeit.py", line 180, in timeit
                 timing = self.inner(it, self.timer)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^
               File "<timeit-src>", line 6, in inner
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/benchmarks/time.py", line 90, in func
                 self.func(*param)
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/benchmarks/moi_curvilinear.py", line 50, in time_load_data_3d
                 for j in range(min(ds.coords["deptht"].size, 2)):
                                    ~~~~~~~~~^^^^^^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/xarray/core/coordinates.py", line 927, in __getitem__
                 return self._data[key]
                        ~~~~~~~~~~^^^^^
               File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/xarray/core/dataset.py", line 1357, in __getitem__
                 raise KeyError(message) from e
             KeyError: "No variable named 'deptht'. Did you mean one of ('depth', 'depth_center')?"

[100.00%] ··· ...vilinear.MOICurvilinear.time_pset_execute_3d             failed
[100.00%] ··· ============== =============
              --             chunk / npart
              -------------- -------------
               interpolator   256 / 10000 
              ============== =============
                 XLinear         failed   
              ============== =============
              For parameters: 'XLinear', 256, 10000
              Traceback (most recent call last):
                File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/server.py", line 179, in _run_server
                  _run(run_args)
                File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/run.py", line 72, in _run
                  result = benchmark.do_run()
                           ^^^^^^^^^^^^^^^^^^
                File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/benchmarks/_base.py", line 661, in do_run
                  return self.run(*self._current_params)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/benchmarks/time.py", line 165, in run
                  samples, number = self.benchmark_timing(
                                    ^^^^^^^^^^^^^^^^^^^^^^
                File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/benchmarks/time.py", line 258, in benchmark_timing
                  timing = timer.timeit(number)
                           ^^^^^^^^^^^^^^^^^^^^
                File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/timeit.py", line 180, in timeit
                  timing = self.inner(it, self.timer)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^
                File "<timeit-src>", line 6, in inner
                File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/.pixi/envs/default/lib/python3.12/site-packages/asv_runner/benchmarks/time.py", line 90, in func
                  self.func(*param)
                File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/benchmarks/moi_curvilinear.py", line 76, in time_pset_execute_3d
                  self.pset_execute_3d(interpolator, chunk, npart)
                File "/storage/home/hodgs004/projects/benchmarking/parcels-benchmarks/benchmarks/moi_curvilinear.py", line 71, in pset_execute_3d
                  pset.execute(
                File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/_core/particleset.py", line 438, in execute
                  self._kernel.execute(self, endtime=next_time, dt=dt)
                File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/_core/kernel.py", line 244, in execute
                  error_func(pset[inds].z, pset[inds].lat, pset[inds].lon)
                File "/storage/home/hodgs004/projects/benchmarking/parcels-vecko/src/parcels/_core/statuscodes.py", line 85, in _raise_grid_searching_error
                  raise GridSearchingError(f"Grid searching failed at (z={z}, lat={y}, lon={x})")
              parcels._core.statuscodes.GridSearchingError: Grid searching failed at (z=array([0.], dtype=float32), lat=array([-21.261051], dtype=float32), lon=array([7.4583335], dtype=float32))

As the error message is different, I think this is orthogonal

@VeckoTheGecko VeckoTheGecko enabled auto-merge (squash) May 6, 2026 13:12
@VeckoTheGecko VeckoTheGecko merged commit b8bab51 into Parcels-code:main May 6, 2026
14 of 15 checks passed
@github-project-automation github-project-automation Bot moved this from Backlog to Done in Parcels development May 6, 2026
@VeckoTheGecko
Copy link
Copy Markdown
Contributor Author

@erikvansebille how should we proceed with #2626 (comment) ? Do you want to continue looking into it?

I don't think it is a blocker for me at the moment, though we will need a running benchmark suite in the next couple weeks sometime

@VeckoTheGecko VeckoTheGecko deleted the push-rttuqnslyyqu branch May 6, 2026 13:35
@erikvansebille
Copy link
Copy Markdown
Member

Hmm, annoying that these benchmark runs continue to fail. I'm having trouble reproducing these runs, as my laptop crashes on the large benchmarks

Perhaps we need to start with smaller/simpler benchmarks? Can you work on performance with a relatively simple script first?

For example the script we use in the Argo tutorial at https://docs.oceanparcels.org/en/main/user_guide/examples/tutorial_Argofloats.html? And then with an without the explicit ds_fields.load()?

@VeckoTheGecko
Copy link
Copy Markdown
Contributor Author

VeckoTheGecko commented May 7, 2026

At the moment I want to work with a stripped down version of Parcels to see how we can best interface with the internals of Xarray - then use that to inform the rest of the work.

We can also work with simpler examples (e.g., items from the test suite or the Argo tutorial) but I think for the moment working with a stripped down version would be more informative

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants