External model support

[pawelwiejkut]

Hallo Team,

Idea: Kindly please add possibility to use external model, via IP connection. I have only one GPU and multiple VM's.

Thank you !

[pierotofy]

What do you mean by "possibility to use external model via IP connection"?

[pawelwiejkut]

basically i would like to separate the model and LTEngine - to do not run everything together, but to be able to connect to the other model in my local network i.e to ollama server

[pierotofy]

Ah, I see; you want to be able to send the LLM prompts via ollama.

While I don't have plans to implement this myself, it might be a useful feature to have the option to do so. Pull requests are welcome. 🙏