[feat] [evaluate] openapi by HymanShi · Pull Request #208 · coze-dev/coze-loop

HymanShi · 2025-09-24T06:16:26Z

What type of PR is this?

openapi for evaluation

Check the PR title.

This PR title match the format: [<type>][<scope>]: <description>. For example: [fix][backend] flaky fix
The description of this PR title is user-oriented and clear enough for others to understand.
Add documentation if the current PR requires user awareness at the usage level.
This PR is written in English. PRs not in English will not be reviewed.

(Optional) Translate the PR title into Chinese.

(Optional) More detailed description for this PR(en: English/zh: Chinese).

en:
zh(optional):

(Optional) Which issue(s) this PR fixes:

(LogID: 2025092312090101009108820949935E9) Co-Authored-By: Coda <coda@bytedance.com>

dsf86 · 2025-09-26T08:44:37Z

idl/thrift/coze/loop/evaluation/coze.loop.evaluation.openapi.thrift

+service EvaluationOpenAPIService {
+    // 评测集接口 (9个)
+    // 1.1 创建评测集
+    CreateEvaluationSetOpenAPIResponse CreateEvaluationSet(1: CreateEvaluationSetOpenAPIRequest req) (api.post = "/open-apis/evaluation/v1/evaluation_sets")


@lsy357 也定义了openapi，但是path不太一致，是不是统一下呢, cc @HymanShi

dsf86 · 2025-09-28T03:44:56Z

idl/thrift/coze/loop/evaluation/coze.loop.evaluation.openapi.thrift

+struct GetEvaluationSetOpenAPIRequest {
+    1: required string workspace_id (api.js_conv="true", go.tag='json:"workspace_id"')
+    2: required string evaluation_set_id (api.path = "evaluation_set_id")
+    3: optional bool include_deleted


这个include_deleted为什么要加呢~

dsf86 · 2025-09-28T03:46:32Z

idl/thrift/coze/loop/evaluation/coze.loop.evaluation.openapi.thrift

+    2: required string name (vt.min_size = "1", vt.max_size = "255")
+    3: optional string description (vt.max_size = "2048")
+    4: optional eval_set.EvaluationSetSchema evaluation_set_schema
+    5: optional string biz_category (vt.max_size = "128")


这个biz_category感觉不需要让用户感知

dsf86 · 2025-09-28T03:56:52Z

idl/thrift/coze/loop/evaluation/domain_openapi/eval_set.thrift

+
+// 评测集
+struct EvaluationSet {
+    1: optional string evaluation_set_id


这里为什么不直接写id呢，还有EvaluationSetVersion里的version_id

dsf86 · 2025-09-28T03:58:12Z

idl/thrift/coze/loop/evaluation/coze.loop.evaluation.openapi.thrift

+
+// 1.1 创建评测集
+struct CreateEvaluationSetOpenAPIRequest {
+    1: required string workspace_id (api.js_conv="true", go.tag='json:"workspace_id"')


这里为什么设置了string呢

Change-Id: I0efae3f7af582460e46ba629c0f470a10c326025

…ationSet, GetEvaluationSet, ListEvaluationSets (LogID: 202509281948170100910941219983125) Co-Authored-By: Coda <coda@bytedance.com>

… convertor (LogID: 202509281948170100910941219983125) Co-Authored-By: Coda <coda@bytedance.com>

…ic to separate methods (LogID: 202509281948170100910941219983125) Co-Authored-By: Coda <coda@bytedance.com>

…rate methods (LogID: 202509281948170100910941219983125) Co-Authored-By: Coda <coda@bytedance.com>

…unified method parameter (LogID: 202509281948170100910941219983125) Co-Authored-By: Coda <coda@bytedance.com>

…error tracking (LogID: 202509281948170100910941219983125) Co-Authored-By: Coda <coda@bytedance.com>

…DTO/xxxDTO2DO convention (LogID: 202509281948170100910941219983125) Co-Authored-By: Coda <coda@bytedance.com>

Change-Id: Ibfa77af15a79fc83b6fb5adc44dd132682fe1c05

…rchitecture (LogID: 20250928231227010091094121028E1E4) Co-Authored-By: Coda <coda@bytedance.com>

Change-Id: I54219f18349d50551302a3129d08ea25f1e15fa1

Change-Id: I4fc1842aea80465df2bfe1d9c7044905b7380cf1

codecov · 2025-09-28T16:23:15Z

Codecov Report

❌ Patch coverage is 0% with 1040 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...tion/convertor/evaluation_set/openapi_convertor.go	0.00%	447 Missing ⚠️
...s/evaluation/application/evaluation_openapi_app.go	0.00%	247 Missing ⚠️
...ndler/coze/loop/apis/evaluation_open_apiservice.go	0.00%	176 Missing ⚠️
backend/api/router/coze/loop/apis/middleware.go	0.00%	84 Missing ⚠️
...ackend/api/router/coze/loop/apis/coze.loop.apis.go	0.00%	46 Missing ⚠️
...es/evaluation/infra/metrics/evaluation_set/emit.go	0.00%	30 Missing ⚠️
...nd/modules/evaluation/infra/rpc/foundation/auth.go	0.00%	5 Missing ⚠️
backend/pkg/kitexutil/kitexutil.go	0.00%	5 Missing ⚠️

❌ Your patch check has failed because the patch coverage (0.00%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

@@            Coverage Diff             @@
##             main     #208      +/-   ##
==========================================
- Coverage   62.76%   61.55%   -1.22%     
==========================================
  Files         479      483       +4     
  Lines       52927    53963    +1036     
==========================================
- Hits        33221    33218       -3     
- Misses      17374    18412    +1038     
- Partials     2332     2333       +1

Flag	Coverage Δ
unittests	`61.55% <0.00%> (-1.22%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...nd/modules/evaluation/infra/rpc/foundation/auth.go	`0.00% <0.00%> (ø)`
backend/pkg/kitexutil/kitexutil.go	`0.00% <0.00%> (ø)`
...es/evaluation/infra/metrics/evaluation_set/emit.go	`20.00% <0.00%> (-24.00%)`	⬇️
...ackend/api/router/coze/loop/apis/coze.loop.apis.go	`0.00% <0.00%> (ø)`
backend/api/router/coze/loop/apis/middleware.go	`2.17% <0.00%> (-0.24%)`	⬇️
...ndler/coze/loop/apis/evaluation_open_apiservice.go	`0.00% <0.00%> (ø)`
...s/evaluation/application/evaluation_openapi_app.go	`0.00% <0.00%> (ø)`
...tion/convertor/evaluation_set/openapi_convertor.go	`0.00% <0.00%> (ø)`

... and 1 file with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d027d8b...01f51b5. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Change-Id: Ic99ec4e867794d8f6d98c1d4bcad118158bb6969

Change-Id: I7d83bcded9ee778cd4baf63b2682053b88f42b44

Change-Id: Icbeb24eeebd9d6d5d67e1038b0f4acfce9d5759a

Change-Id: I7a78d11c6b3bdd59bb9a08d5cd9daf893c8afa15

Change-Id: I264264a0e8bbf776341fc2149377b7ba934d1c61

Change-Id: Iac21bc6194af354d9c4ad3a0e1a528cccd0f0c4f

Change-Id: I7972ba4c3b4b812652b86187366c2fdd03c7763e

Change-Id: I673a5dc343764edf228ec5d8a8219f0659d8278d

Change-Id: Ie1e50446c90172aea7292f134b117a37a84f0bce

Change-Id: Ia048d5d8836418597fa6ec48c602fbf48e197643

Change-Id: Ic21981040de379923d8a216a0f8fcf9b543072e6

Change-Id: I86989136f4b8baa51b399df4bed50b3edef4018d

HymanShi and others added 7 commits September 22, 2025 19:46

add evaluation's openapi idl file

302866d

add evaluation's openapi idl file

5c721c4

feat: [Coda] 基于设计文档生成评测模块完整OpenAPI接口定义，包含16个核心接口和公共数据结构

a2d0010

(LogID: 2025092312090101009108820949935E9) Co-Authored-By: Coda <coda@bytedance.com>

add evaluation's openapi idl file

9c0ca15

add evaluation's openapi idl file

07cad62

feat: [Coda] 为评测模块OpenAPI所有接口添加workspace_id参数，支持工作空间级别的资源隔离

61c051e

(LogID: 2025092312090101009108820949935E9) Co-Authored-By: Coda <coda@bytedance.com>

feat: [Coda] 调整评测模块OpenAPI接口备注格式，将行内备注改为另起一行提升代码可读性

bec5dbe

(LogID: 2025092312090101009108820949935E9) Co-Authored-By: Coda <coda@bytedance.com>

mocayo changed the title ~~Feat/eval openapi~~ [feat] [evaluate] openapi Sep 24, 2025

dsf86 reviewed Sep 26, 2025

View reviewed changes

dsf86 reviewed Sep 28, 2025

View reviewed changes

dsf86 and others added 12 commits September 28, 2025 14:34

fix

a8f6568

Change-Id: I0efae3f7af582460e46ba629c0f470a10c326025

feat: [Coda] implement evaluation OpenAPI interfaces with CreateEvalu…

73d6a07

…ationSet, GetEvaluationSet, ListEvaluationSets (LogID: 202509281948170100910941219983125) Co-Authored-By: Coda <coda@bytedance.com>

fix: [Coda] correct ContentType and Status enum conversion in OpenAPI…

d54f0c1

… convertor (LogID: 202509281948170100910941219983125) Co-Authored-By: Coda <coda@bytedance.com>

refactor: [Coda] extract ContentType and DisplayFormat conversion log…

17de498

…ic to separate methods (LogID: 202509281948170100910941219983125) Co-Authored-By: Coda <coda@bytedance.com>

refactor: [Coda] extract EvaluationSetStatus conversion logic to sepa…

231c6c5

…rate methods (LogID: 202509281948170100910941219983125) Co-Authored-By: Coda <coda@bytedance.com>

refactor: [Coda] simplify OpenAPIEvaluationSetMetrics interface with …

4f0e0f2

…unified method parameter (LogID: 202509281948170100910941219983125) Co-Authored-By: Coda <coda@bytedance.com>

feat: [Coda] enhance OpenAPI metrics with response time and detailed …

d84f80e

…error tracking (LogID: 202509281948170100910941219983125) Co-Authored-By: Coda <coda@bytedance.com>

refactor: [Coda] standardize convertor method naming to follow xxxDO2…

bac91ca

…DTO/xxxDTO2DO convention (LogID: 202509281948170100910941219983125) Co-Authored-By: Coda <coda@bytedance.com>

fix

24cba69

Change-Id: Ibfa77af15a79fc83b6fb5adc44dd132682fe1c05

feat: [Coda] implement 6 evaluation set OpenAPI interfaces with DDD a…

17fee29

…rchitecture (LogID: 20250928231227010091094121028E1E4) Co-Authored-By: Coda <coda@bytedance.com>

fix

243c07a

Change-Id: I54219f18349d50551302a3129d08ea25f1e15fa1

fix

9b0cc0d

Change-Id: I4fc1842aea80465df2bfe1d9c7044905b7380cf1

dsf86 added 4 commits September 29, 2025 00:23

fix

8d37796

Change-Id: Ic99ec4e867794d8f6d98c1d4bcad118158bb6969

fix

5802023

Change-Id: I7d83bcded9ee778cd4baf63b2682053b88f42b44

fix

b4464f6

Change-Id: Icbeb24eeebd9d6d5d67e1038b0f4acfce9d5759a

fix

2b0c437

Change-Id: I7a78d11c6b3bdd59bb9a08d5cd9daf893c8afa15

dsf86 added 8 commits September 29, 2025 14:26

fix

b1d7f28

Change-Id: I264264a0e8bbf776341fc2149377b7ba934d1c61

fix

eaeb1db

Change-Id: Iac21bc6194af354d9c4ad3a0e1a528cccd0f0c4f

fix

f09751a

Change-Id: I7972ba4c3b4b812652b86187366c2fdd03c7763e

fix

b043cb1

Change-Id: I673a5dc343764edf228ec5d8a8219f0659d8278d

fix

cdf914b

Change-Id: Ie1e50446c90172aea7292f134b117a37a84f0bce

fix

055ed40

Change-Id: Ia048d5d8836418597fa6ec48c602fbf48e197643

fix

85b62ed

Change-Id: Ic21981040de379923d8a216a0f8fcf9b543072e6

fix

01f51b5

Change-Id: I86989136f4b8baa51b399df4bed50b3edef4018d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[feat] [evaluate] openapi#208

[feat] [evaluate] openapi#208
HymanShi wants to merge 31 commits intomainfrom
feat/eval_openapi

HymanShi commented Sep 24, 2025

Uh oh!

dsf86 Sep 26, 2025

Uh oh!

dsf86 Sep 28, 2025

Uh oh!

dsf86 Sep 28, 2025

Uh oh!

dsf86 Sep 28, 2025 •

edited

Loading

Uh oh!

dsf86 Sep 28, 2025

Uh oh!

codecov bot commented Sep 28, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

HymanShi commented Sep 24, 2025

What type of PR is this?

Check the PR title.

(Optional) Translate the PR title into Chinese.

(Optional) More detailed description for this PR(en: English/zh: Chinese).

(Optional) Which issue(s) this PR fixes:

Uh oh!

dsf86 Sep 26, 2025

Choose a reason for hiding this comment

Uh oh!

dsf86 Sep 28, 2025

Choose a reason for hiding this comment

Uh oh!

dsf86 Sep 28, 2025

Choose a reason for hiding this comment

Uh oh!

dsf86 Sep 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dsf86 Sep 28, 2025

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Sep 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dsf86 Sep 28, 2025 •

edited

Loading

codecov bot commented Sep 28, 2025 •

edited

Loading