Skip to content

refactor(pathfinder): evaluate replacing spawned_process_runner with dedicated subprocess entrypoints #1771

@cpcloud

Description

@cpcloud

Context

In PR #1768 we switched the CTK canary probe away from multiprocessing.get_context(\"spawn\") and back to a dedicated python -m ... subprocess entrypoint, while keeping cuda.pathfinder._utils.spawned_process_runner in place for the callable-based isolation paths used by tests.

During review, a follow-on idea came up: could cuda_pathfinder use the same fully isolated subprocess pattern for those tests as well, and potentially eliminate spawned_process_runner.py entirely?

Motivation

Right now cuda_pathfinder has two different subprocess-isolation mechanisms:

  • dedicated module subprocesses for the canary probe
  • multiprocessing spawn via spawned_process_runner.py for test helpers / callable execution

This works today, but it means we have two different models to reason about when debugging subprocess behavior, import resolution, loader-state isolation, and script-entrypoint behavior.

Question to investigate

Would it be better to replace the remaining spawned_process_runner.py usage in cuda_pathfinder with the same dedicated-subprocess approach used by the canary probe?

Things to evaluate:

  • whether the test helpers can be expressed cleanly as module/file entrypoints instead of spawned Python callables
  • whether the fully isolated subprocess approach preserves the same loader-state guarantees that motivated spawn in the first place
  • whether this would simplify maintenance by making production and test isolation use one mechanism
  • whether there are places where callable-based spawn is still meaningfully better and should be kept

Scope

This is a follow-on refactor / design issue, not a blocker for #1768.

Metadata

Metadata

Assignees

Labels

P0High priority - Must do!cuda.pathfinderEverything related to the cuda.pathfinder module

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions