Generic numeric debugging

[metascroy]

Differential Revision: D103956056

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19317

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 5 New Failures, 6 Pending

As of commit 7ad210a with merge base 0f9de6a ():

NEW FAILURES - The following jobs have failed:

• pull / test-coreml-bc-macos (macos-m1-stable) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• pull / unittest / linux / linux-job (gh)
  examples/models/test/test_export.py::ExportTest::test_mv3_export_to_executorch
• pull / unittest / windows / windows-job (gh)
  Process completed with exit code 2.
• pull / unittest-editable / linux / linux-job (gh)
  examples/models/test/test_export.py::ExportTest::test_mv3_export_to_executorch
• pull / unittest-editable / windows / windows-job (gh)
  Process completed with exit code 2.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[meta-codesync]

@metascroy has exported this pull request. If you are a Meta employee, you can view the originating Diff in D103956056.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[meta-codesync]

@metascroy has imported this pull request. If you are a Meta employee, you can view this in D103956056.

[digantdesai]

what happens if the delegate have to fuse some candidate nodes? Try lowering conv2d-->batch_norm, where XNNPACK doesn't support standalone batch_norm IIRC.
Also how is it better than forced-single-op-partitions?

[metascroy]

This is primarily to help RL in a numeric debugging investigation. I may clean it up to make it a generic utility if they find it useful, but will likely go through design review to get input from others if I do that.

To answer your question on fused candidates: this is tested with CoreML's quantized linear pattern [dequantize -> linear]. In that case, we tap the intermediate output after linear node, which is actually a quantized linear in both eager (from QDQ pattern) and CoreML (from its internal fusion). In the [conv2d-->batch_norm] case, I'd have to check. Tapping batch norm should mean we want the output of batch_norm, i.e., the intermediate output after batchnorm, which can be the result of a fused [conv2d-->batch_norm] op. If we did forced single-op partitions on con2d and batch_norm separately, we wouldn't get the fusion.

how is it better than forced-single-op-partitions

One reason is single-op partitions destroy fusion that backends do. Here we tap intermediates, so as long as we tap the final intermediate after a fusion pattern we should be good.

A second reason is this approach does keep the same big delegate blob, just with extra outputs. In CoreML's case that means it will still be routed to ANE, whereas if you break up into single op partitions, it will very likely reroute to CPU b/c they are small.