Skip to content

Fix singleton lifetime issues at process exit#3555

Open
dhiltgen wants to merge 1 commit into
ml-explore:mainfrom
dhiltgen:pr/singleton-leaks
Open

Fix singleton lifetime issues at process exit#3555
dhiltgen wants to merge 1 commit into
ml-explore:mainfrom
dhiltgen:pr/singleton-leaks

Conversation

@dhiltgen
Copy link
Copy Markdown
Contributor

Proposed changes

Split out from #3019

Leak the IO ThreadPool singletons and CPU CompilerCache using the same process-lifetime pattern already used by the Scheduler singleton.

The CompilerCache owns dlopen handles for JIT shared libraries. Destroying it during static teardown can dlclose generated code while stream worker threads may still be winding down. The IO loader thread pools have the same shutdown-order risk on Windows CRT teardown. These objects are process-lifetime infrastructure, and the OS reclaims their resources at exit.

Changes:

  • Leak CompilerCache so JIT libraries remain mapped through process exit
  • Leak IO ThreadPool singletons to avoid teardown-order races
  • Clarify the Scheduler singleton comment that documents this pattern

Checklist

Put an x in the boxes that apply.

  • I have read the CONTRIBUTING document
  • I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the necessary documentation (if needed)

Leak the IO ThreadPool singletons and CPU CompilerCache using the same process-lifetime pattern already used by the Scheduler singleton.

The CompilerCache owns dlopen handles for JIT shared libraries. Destroying it during static teardown can dlclose generated code while stream worker threads may still be winding down. The IO loader thread pools have the same shutdown-order risk on Windows CRT teardown. These objects are process-lifetime infrastructure, and the OS reclaims their resources at exit.

Changes:
- Leak CompilerCache so JIT libraries remain mapped through process exit
- Leak IO ThreadPool singletons to avoid teardown-order races
- Clarify the Scheduler singleton comment that documents this pattern
Copy link
Copy Markdown
Collaborator

@zcbenz zcbenz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants