Fixes #875: add out-of-core dask CPU viewshed#897
Merged
brendancol merged 5 commits intomasterfrom Feb 25, 2026
Merged
Conversation
Add a dask+numpy backend for viewshed() using a three-tier strategy: A. max_distance specified → extract spatial window, compute, run exact R2 angular sweep on the small numpy window. B. Full R2 fits in memory (280 bytes/cell < 50% available RAM) → compute the full dask array, run R2 directly. C. Otherwise → horizon-profile distance sweep algorithm that processes cells in Chebyshev-ring order with an LRU chunk cache, keeping memory bounded regardless of grid size. Also adds a max_distance parameter to viewshed() for all backends, a memory guard for Tier C, and four new dask-specific tests.
- Extract max_distance window logic into _viewshed_windowed() that works for numpy, cupy, dask+numpy, and dask+cupy uniformly - Handle dask+cupy in _viewshed_dask(): Tier B computes to cupy and uses GPU RTX when available; Tier C converts cupy chunks to numpy for the distance-sweep fallback - Add tests: numpy max_distance, max_distance-vs-full comparison (numpy + cupy parametrized)
…ndow _viewshed_windowed was allocating np.full((H, W), ...) for the output before wrapping it as dask — instant OOM on a 30TB input even with max_distance set. Now for dask inputs the output is built chunk-by-chunk: overlapping chunks get a concrete numpy block, all others are lazy da.full blocks that consume no memory until materialized. Adds test_viewshed_dask_max_distance_lazy_output which creates a 100k x 100k (80GB) dask raster and verifies the output stays lazy.
The window extraction uses a square (Chebyshev) region but max_distance is a Euclidean radius. Add a circular mask after computing the window result so cells at the corners beyond max_distance are set to INVISIBLE. Update tests to account for the circular boundary.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
viewshed()with a three-tier strategy:max_distanceset): extract spatial window from dask array, compute only relevant chunks, run exact R2 angular sweep on the small numpy windowmax_distanceparameter toviewshed()for all backendsMemoryErrorbefore allocating if output grid exceeds 80% of available RAMCloses #875
Test plan
pytest xrspatial/tests/test_viewshed.py -x -v)max_distance