Skip to content

Fix Conv3d w8a8 quantization rank mismatch#2712

Open
tritolol wants to merge 2 commits into
apple:mainfrom
tritolol:fix/conv3d-w8a8-quantization-rank-mismatch
Open

Fix Conv3d w8a8 quantization rank mismatch#2712
tritolol wants to merge 2 commits into
apple:mainfrom
tritolol:fix/conv3d-w8a8-quantization-rank-mismatch

Conversation

@tritolol
Copy link
Copy Markdown
Contributor

Summary

Test plan

  • New test test_linear_quantizer_conv3d_w8a8 passes
  • Existing quantization conversion tests unaffected

🤖 Generated with Claude Code

Scale/zero-point expansion for conv weights only handled rank-4 (Conv2d).
Generalize to rank >= 4 so Conv3d (rank-5) weights are also covered.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>


# region LinearQuantizer Conv3d
def test_linear_quantizer_conv3d_w8a8():
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test fails even with your fix.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review! The test was failing on macOS because verify_model_outputs hardcodes "input_1" as the predict input key ([conversion_utils.py:107]), but the forward parameter x caused CoreML to name the input "x_1" instead. Fixed by renaming the parameter to input_1 to match the convention used by the rest of the test suite.

verify_model_outputs hardcodes "input_1" as the predict key on macOS.
Renaming the forward parameter to input_1 ensures CoreML names the
model input consistently.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

coremltools.optimize.torch.quantization fails to handle w8a8 conv3d quantization properly

2 participants