Skip to content

Add managed-memory advise, prefetch, and discard-prefetch on Buffer#1775

Draft
rparolin wants to merge 7 commits intoNVIDIA:mainfrom
rparolin:rparolin/managed_mem_advise_prefetch
Draft

Add managed-memory advise, prefetch, and discard-prefetch on Buffer#1775
rparolin wants to merge 7 commits intoNVIDIA:mainfrom
rparolin:rparolin/managed_mem_advise_prefetch

Conversation

@rparolin
Copy link
Collaborator

@rparolin rparolin commented Mar 17, 2026

Summary

Add managed-memory range controls to cuda.core.Buffer:

  • Buffer.advise(...)
  • Buffer.prefetch(...)
  • Buffer.discard_prefetch(...)

This addresses #1332.

Details

These operations are implemented on Buffer because the CUDA driver APIs apply to a memory range, not specifically to
ManagedMemoryResource. That allows the new methods to work for both:

  • buffers allocated from ManagedMemoryResource
  • externally wrapped managed allocations

The implementation adds managed-memory detection for buffers, normalizes device/host/NUMA location inputs, and forwards to the existing
driver bindings for cuMemAdvise, cuMemPrefetchAsync, and cuMemDiscardAndPrefetchBatchAsync.

Tests

Adds coverage for:

  • advise/prefetch/discard-prefetch on managed-memory buffers
  • support for externally wrapped managed allocations
  • rejection on non-managed buffers
  • invalid input combinations
  • observable driver-side effects via cuMemRangeGetAttribute(...)

Closes #1332

@copy-pr-bot
Copy link
Contributor

copy-pr-bot bot commented Mar 17, 2026

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@rparolin rparolin requested a review from Andy-Jost March 17, 2026 00:41
@rparolin rparolin self-assigned this Mar 17, 2026
@rparolin rparolin added this to the cuda.core v0.7.0 milestone Mar 17, 2026
@rparolin rparolin marked this pull request as ready for review March 17, 2026 00:45
@rparolin rparolin marked this pull request as draft March 17, 2026 00:45
@rparolin rparolin changed the title wip Add managed-memory advise, prefetch, and discard-prefetch on Buffer Mar 17, 2026
@rparolin rparolin marked this pull request as ready for review March 17, 2026 00:57
@github-actions
Copy link

@rparolin
Copy link
Collaborator Author

/ok to test

@jrhemstad
Copy link

question: Does making these member functions of the Buffer type preclude this functionality for allocations that weren't created through the Buffer type? Did we consider making these free functions instead of member functions on the Buffer type?

@rparolin
Copy link
Collaborator Author

rparolin commented Mar 17, 2026

question: Does making these member functions of the Buffer type preclude this functionality for allocations that weren't created through the Buffer type? Did we consider making these free functions instead of member functions on the Buffer type?

I'm moving this back into draft. We discussed in our team meeting because I was already hesitant as Buffer is becoming a 'God object' with the functionality is gaining. We were going to explore alternatives. Free functions sounds like a good alternative to explore.

@rparolin rparolin marked this pull request as draft March 17, 2026 19:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support managed memory advise, prefetch, and discard-prefetch

2 participants