is it possible to use cuda

[twmht]

Hi,

is it possible to use cuda for inference?

[staghado]

It is possible : refer to the GGML library.

[twmht]

yeah i know ggml supports cuda, what i mean is do you have any plan to add options(https://github.com/staghado/vit.cpp?tab=readme-ov-file#run) for cuda?

[staghado]

I can add it if that's something you need. One issue is that i do not have a local GPU so it will be hard to test. would you like to test? you can also open a PR if you like!

[twmht]

@staghado

unfortunately i am not familir with ggml now, but i can test if you can help to submit a PR:)