Skip to content

user provided bound for torchtrt compile when export dimension is unb…#4213

Open
apbose wants to merge 6 commits intomainfrom
abose/torchTRT_dynamic_user_shape_bounds
Open

user provided bound for torchtrt compile when export dimension is unb…#4213
apbose wants to merge 6 commits intomainfrom
abose/torchTRT_dynamic_user_shape_bounds

Conversation

@apbose
Copy link
Copy Markdown
Collaborator

@apbose apbose commented Apr 27, 2026

This PR validates user bounds against any finite exporter bounds, with three branches:

  • Upper overflow (user_max > exp_max) → ValueError. TRT engine profile follows the exporter; shapes above exp_max would be rejected at runtime anyway.
  • Lower underflow (user_min < exp_min) → ValueError, except the (user_min=1, exp_min=2) case which is just PyTorch's 0/1-specialization artifact (Dim(min=1) silently becomes 2 in ShapeEnv), this particular case emits a warning.
  • Subset (user range inside exporter range) → warning that the engine profile will be widened to the exporter's bounds.
  • Dim.DYNAMIC (unbounded exporter upper) → no check, user fills the gap (the intended use).

@meta-cla meta-cla Bot added the cla signed label Apr 27, 2026
@github-actions github-actions Bot added component: tests Issues re: Tests component: core Issues re: The core compiler component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels Apr 27, 2026
@github-actions github-actions Bot requested a review from lanluo-nvidia April 27, 2026 19:27
@apbose apbose force-pushed the abose/torchTRT_dynamic_user_shape_bounds branch from 4d1ff02 to eec2c04 Compare April 27, 2026 23:35
@apbose
Copy link
Copy Markdown
Collaborator Author

apbose commented Apr 30, 2026

CI index put cases failing. Looking at it.

@apbose apbose force-pushed the abose/torchTRT_dynamic_user_shape_bounds branch from 0fe114e to 2905ff6 Compare April 30, 2026 01:05
…lders; sync assertions with refactored bound-mismatch messages
@apbose apbose requested a review from narendasan May 6, 2026 21:09
@narendasan
Copy link
Copy Markdown
Collaborator

This PR is only for externally exported programs?

@apbose
Copy link
Copy Markdown
Collaborator Author

apbose commented May 6, 2026

@narendasan yeah it addresses (torch_tensorrt.dynamo.compile(exported_program, ...)). The flow for torch_tensorrt.compile(model, inputs=..) should be unaffected, since in that case export bounds and input bounds should be same

@apbose apbose force-pushed the abose/torchTRT_dynamic_user_shape_bounds branch from dee44b1 to 22728b3 Compare May 6, 2026 22:15
Copy link
Copy Markdown
Collaborator

@narendasan narendasan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should use the user bounds in the narrow case

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla signed component: api [Python] Issues re: Python API component: core Issues re: The core compiler component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths component: tests Issues re: Tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants