Support img2img

[yocontra]

Problem description

Happy to send a PR attempt at this with a little guidance/grace!

Proposed solution

Support for something like Hyper-SDXL Raw that can do img2img

Alternative solutions

No response

Benefits to React Native ExecuTorch

People want this.

Additional context

No response

[msluszniak]

Hi @yocontra,

We are very excited that you want to contribute to our library. I will be very happy to help you with the PR that will introduce the support for img2img :))

[yocontra]

Great - I'm getting img2img working standalone right now using apple's diffusion tools (sdxl -> coreml conversion) then I'll figure out how to port that to executorch. I'm using executorch for TTS and LLM in the same application already so having it all under one roof would be great :) Are there any models/code paths in this project that have a hybrid "use this coreml path on ios, do this other thing on android" type of approach I can copy? Or would that be a new type of branch.

[msluszniak]

Indeed, there are some places with such paths. We currently experimenting with exports on different backends that utilze GPUs / NPUs. CoreML seems to be a mature backend that provides fairly easy exports using ExecuTorch. But analogical tools for Android, e.g. Vulkan doesn't work as smooth. Because of that, we provide highly optimized CoreML exports on iOS, and fallback XNNPack exports for other Android. Example of these are in this PR: https://github.com/software-mansion/react-native-executorch/pull/940/changes#diff-2a5630af284e2c061f7c2e3c723167a4f57ff32ffe8530824361c8c280b232b4R377