Skip to content

issue/160: 梳理 InferEngine 相关接口 #161

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
voltjia merged 8 commits into from
Dec 26, 2025
Merged

issue/160: 梳理 InferEngine 相关接口 #161

voltjia merged 8 commits into from
Dec 26, 2025

Conversation

@voltjia
Copy link
Collaborator

@voltjia voltjia commented Dec 25, 2025

image

@voltjia voltjia linked an issue Dec 25, 2025 that may be closed by this pull request
[DEV] 将 cpp.LlamaForCausalLM 提出，变为 infinilm.infer_engine.InferEngine #160
Closed
@voltjia voltjia requested a review from a team December 26, 2025 08:55
@voltjia voltjia changed the title issue/160: 将 cpp.LlamaForCausalLM 提出，变为 infinilm.infer_engine.InferEngine，并将 Config 构造逻辑拆分至 AutoConfig 中 issue/160: 梳理 infinilm.infer_engine.InferEngine 相关接口 Dec 26, 2025
@voltjia
Copy link
Collaborator Author

voltjia commented Dec 26, 2025

合入此 PR 前请先确保 InfiniTensor/InfiniCore#861 已合入。

@voltjia
Copy link
Collaborator Author

voltjia commented Dec 26, 2025

image

@voltjia voltjia added the enhancement New feature or request label Dec 26, 2025
This was linked to issues Dec 26, 2025
[DEV] 将 random_sample 放入模型中 #152
Closed
[DEV] 梳理 generate 接口 #154
Closed
@voltjia
Copy link
Collaborator Author

voltjia commented Dec 26, 2025

python ./test/bench/test_benchmark.py --nvidia /data/shared/models/TinyLlama -1.1B-Chat-v1.0 --bench ceval 的部分输出如下：

image image

@voltjia voltjia changed the title issue/160: 梳理 infinilm.infer_engine.InferEngine 相关接口 issue/160: 梳理 InferEngine 相关接口 Dec 26, 2025
@voltjia
Copy link
Collaborator Author

voltjia commented Dec 26, 2025

image

@voltjia
Copy link
Collaborator Author

voltjia commented Dec 26, 2025

python ./test/bench/test_benchmark.py --nvidia /data/shared/models/TinyLlama -1.1B-Chat-v1.0 --bench ceval 的部分输出如下：

image image

PanZezhong1725
PanZezhong1725 requested changes Dec 26, 2025
View reviewed changes
@voltjia
Copy link
Collaborator Author

voltjia commented Dec 26, 2025

image

@voltjia
Copy link
Collaborator Author

voltjia commented Dec 26, 2025

python ./test/bench/test_benchmark.py --nvidia /data/shared/models/TinyLlama -1.1B-Chat-v1.0 --bench ceval 的部分输出如下：

image image

@voltjia voltjia merged commit 34c0770 into main Dec 26, 2025
@voltjia voltjia deleted the issue/160 branch December 26, 2025 11:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wooway777 wooway777 wooway777 approved these changes

@PanZezhong1725 PanZezhong1725 Awaiting requested review from PanZezhong1725

@pengcheng888 pengcheng888 Awaiting requested review from pengcheng888

Assignees

No one assigned

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

4 participants

@voltjia @wooway777 @PanZezhong1725