feat: Added a backpressure mechanism to shortestPath #9576
+19
−2
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The backpressure mechanism should prevent dropping possible paths from the priority queue, by not filling the queue when there is too much in it.
Description
#9333 Here an issue was fixed that could in some cases cause an out of memory when doing a kshortestPath search. In short if the graph is very intertwined with a lot of edges between a smaller set of nodes (currently 140k edges over 10k nodes) a simple search with weights active can cause OOM due to the adjacency map overflowing the priorityqueue.
This was somewhat fixed by #9382 which introduced an interesting bug that only now got noticed.
When the MaxFrontierSize is reached a pop is called on the queue, which actually drops the shortest element, meaning that in case of a depth>1 search with active and triggered MaxFrontierSize it is unlikely to get the actual shortest path.
As that is an algorithmic issue and cant really be fixed that easily.
So the PR is aiming at actually slowing the neighbour search down to give the algorithm time to clear the queue off items that are over the bounds of depth or maxweight. In case of no constraints on the paths it is expected to have a high memory usage hence usage of MaxFrontierSize will in any way cause misses.
Checklist
Conventional Commits syntax, leading
with
fix:,feat:,chore:,ci:, etc.