issue/350 - add ChatGLM causal LM model support#353
Open
JoeZhang-0x000 wants to merge 2 commits intoInfiniTensor:mainfrom
Open
issue/350 - add ChatGLM causal LM model support#353JoeZhang-0x000 wants to merge 2 commits intoInfiniTensor:mainfrom
JoeZhang-0x000 wants to merge 2 commits intoInfiniTensor:mainfrom
Conversation
- Add GLM4 model config adapter (csrc/models/glm4/) - Add partial rotary support (llama_utils.hpp, llama_attention.*) - Add forward_naive + GLM4 post-norm support (llama_decoder_layer.*) - Add RoPE algo selection for GLM4 (llama_model.cpp) - Add GLM4 special model creation path (rank_worker.cpp) - Register glm4 in config_factory.cpp and auto_config.py - Add GLM4 weight remapping (gate_up_proj split) in modeling_utils.py
- Add ChatGLM model config adapter (csrc/models/chatglm/) - Extend rank_worker.cpp condition to include chatglm - Register chatglm in config_factory.cpp and auto_config.py - Add ChatGLM weight remapping (key rename + QKV/FFN split) in modeling_utils.py - Update test_infer.py for ChatGLM tokenizer special token handling
Collaborator
|
请先处理glm4的#352 |
Collaborator
在当前分支模型输出为空 但在之前的三合一分支正常： |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
csrc/models/chatglm/)rank_worker.cppcondition to include"chatglm""chatglm"inconfig_factory.cppclassic_models list andauto_config.pymodeling_utils.pytest_infer.pyfor ChatGLM tokenizer special token handlingNote: This branch is based on
issue/349(GLM4 support) since ChatGLM depends on GLM4's infrastructure changes (partial rotary, RoPE algo, rank_worker special path). Please merge #352 first.Closes #350
Parent issue: #332