Skip to content

Conversation

@gongchensu
Copy link
Collaborator

…ript.
使用方式:
python transformers_simple_inference.py --model_path /data-aisoft/mechdancer/models/9G7B_QW/
用transformers库或任何其它库跑九格7B的hack——将九格模型强行视为qwen2

  1. 创建一个新的目录,并在目录里为原模型目录下所有文件创建link(避免weight拷贝)
  2. 重新编辑config.json
  3. 跑模型时传入新的目录
  4. 注意tokenizer还要用九格原来的,所以创建tokenizer是需要打开"trust_remote_code=True"
    9g7B改qwen2的config:
    { "architectures": [ "Qwen2ForCausalLM" ], "attention_dropout": 0.0, "bos_token_id": 1, "eos_token_id": [ 2, 73440 ], "hidden_act": "silu", "hidden_size": 3584, "initializer_range": 0.02, "intermediate_size": 18944, "max_position_embeddings": 32768, "max_window_layers": 28, "model_type": "qwen2", "num_attention_heads": 28, "num_hidden_layers": 28, "num_key_value_heads": 28, "rms_norm_eps": 1e-06, "rope_theta": 1000000.0, "sliding_window": null, "tie_word_embeddings": false, "torch_dtype": "bfloat16", "transformers_version": "4.44.0", "use_cache": true, "use_mrope": false, "use_sliding_window": false, "vocab_size": 73448 }
image

@gongchensu gongchensu self-assigned this Dec 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant