Describe the feature
After a laborious journey through Apple Notarization, I discovered that the only way to package and ship builds for mac os with larger LLMs is through gguf chunking.
Notarization fails for files larger ≈ 4GB
Loading chunked ggufs already works in LLMUnity/llama.cpp
If you could issue a warning or document this, it might save other OSX developers a lot of headaches.
Ideally, chunking via llama-gguf-split (part of llama.cpp tools) would be integrated into LLMUnity and offered via the LLM/Build Manager.
Describe the feature
After a laborious journey through Apple Notarization, I discovered that the only way to package and ship builds for mac os with larger LLMs is through gguf chunking.
Notarization fails for files larger ≈ 4GB
Loading chunked ggufs already works in LLMUnity/llama.cpp
If you could issue a warning or document this, it might save other OSX developers a lot of headaches.
Ideally, chunking via llama-gguf-split (part of llama.cpp tools) would be integrated into LLMUnity and offered via the LLM/Build Manager.