Upgrade Metal backend to unified GPU/ANE architecture#20
Merged
ChinChangYang merged 1 commit intoios-devfrom Apr 7, 2026
Merged
Upgrade Metal backend to unified GPU/ANE architecture#20ChinChangYang merged 1 commit intoios-devfrom
ChinChangYang merged 1 commit intoios-devfrom
Conversation
…ctor#1148 Replace the separate CoreML and Metal backends with a single unified Metal backend that converts .bin.gz models to CoreML on-the-fly using the vendored katagocoreml library. This enables a per-thread GPU/ANE multiplexer where each NN server thread is dedicated to either GPU (MPSGraph, gpuIdx=0) or ANE (CoreML CPU+ANE, gpuIdx=100). Key changes: - Add metallayers.swift for MPSGraph GPU layer implementations - Rewrite metalbackend.swift for CoreML model loading and inference - Add vendored katagocoreml C++ library (88 files) for native model conversion - Remove coremlbackend.cpp/h/swift and coremlmodel.swift - Update C++ support files (nneval, setup, gtp, commandline) to remove CoreML- specific code paths and the separate CoreML evaluator - Simplify iOS app interface: all models use Metal backend (no more builtIn vs downloaded distinction for backend selection) - Change built-in model from CoreML .mlpackage (b28) to 18b .bin.gz network - Update default_gtp.cfg with Metal backend settings (metalDeviceToUseThread0, metalUseFP16) - iOS defaults to ANE (Neural Engine) for power efficiency; macOS uses GPU - Update ci_post_clone.sh to build katagocoreml and fetch 18b model - Remove CoreML .mlpackage downloads from CI scripts https://claude.ai/code/session_01LLDk6NpbEpchikWcq21ksf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
katagocoremlC++ library for on-the-fly.bin.gz→ CoreML model conversion.mlpackage(b28c512) to an 18-block.bin.gznetwork that the Metal backend converts at runtime, eliminating the need for pre-converted CoreML modelsKey changes
C++ Backend
metalbackend.cpp/h— unified backend withCoreMLConversionnamespace for native model conversion, per-threadMETAL_MUX_GPU(0) /METAL_MUX_ANE(100) multiplexermetalbackend.swift— rewritten for CoreML model loading/inference viaCoreMLComputeHandlemetallayers.swift— new file with MPSGraph GPU layer implementations andMPSGraphModelHandlecpp/external/katagocoreml/— vendored C++ library (88 files) for.bin.gz→.mlpackageconversioncoremlbackend.cpp/h/swift,coremlmodel.swift, and all CoreML-specific code paths innneval,setup,gtp,commandlineiOS App
KataGoHelper.swift— simplified: removedcoremlModelPath/useMetalparams; usesmetalDeviceToUseThread0+metalUseFP16ModelRunnerView.swift— unified model loading (no more builtIn vs downloaded backend distinction)NeuralNetworkModel.swift— built-in model is now "Built-in KataGo 18b Network" (.bin.gz)default_gtp.cfg— Metal backend config (metalDeviceToUseThread0=100,metalUseFP16=true)CI/Build
ci_post_clone.sh— installs cmake/ninja/protobuf/abseil, builds katagocoreml for all platforms, fetches 18b modelbuild.yml— removed CoreML.mlpackagedownload/test stepsmetallayers.swift, removed CoreML files,USE_COREML_BACKEND→USE_METAL_BACKEND, added katagocoreml linker flagsTest plan
.bin.gzmodels work through the Metal backendhttps://claude.ai/code/session_01LLDk6NpbEpchikWcq21ksf