-
Notifications
You must be signed in to change notification settings - Fork 101
Open
Description
我下载Huggingface开源的TeleChat-1B模型和模型介绍中的推理示例代码,结果输出与问题无关,换个问题输出就乱码,麻烦看下啦,谢谢
代码:
import os
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig
os.environ["CUDA_VISIBLE_DEVICES"] = '0'
tokenizer = AutoTokenizer.from_pretrained('*/models/TeleChat-1B', trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained('*/models/TeleChat-1B', trust_remote_code=True, device_map="auto", torch_dtype=torch.float16)
question="<_user>生抽与老抽的区别?<_bot>"
context_ids = tokenizer(question, return_tensors="pt")
output = model.generate(context_ids["input_ids"].to(0), do_sample=False, max_length=1024)
print(output)
answer = tokenizer.decode(output[0].tolist()).split('<_bot>')[-1]
print(answer)
结果如下图:
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
