Fix validate consistency #7679
Open
Fix validate consistency #7679
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
感谢你贡献飞桨文档，文档预览构建中，Docs-New 跑完后即可预览，预览链接：http://preview-pr-7679.paddle-docs-preview.paddlepaddle.org.cn/documentation/docs/zh/api/index_cn.html |
📚 本次 PR 文档预览链接（点击展开）
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes validation consistency issues in the PyTorch to PaddlePaddle API documentation conversion tools. The changes improve the API difference validation script, refactor the API discovery logic to recursively scan all markdown files, and reorganize API documentation by moving files from category-specific subdirectories to more appropriate locations.
Changes:
- Enhanced validation script with overloaded API support and optimized lookup performance using dictionary mapping
- Refactored API discovery to recursively scan all markdown files regardless of directory structure
- Reorganized API documentation files, moving transformers APIs from
torch_more_args/paddle_more_args/otherstoinvok_only_diff/args_name_diff/composite_implement - Updated various API signatures with missing asterisks (*) to denote keyword-only parameters
Reviewed changes
Copilot reviewed 153 out of 153 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| validate_api_difference_consistency.py | Added OVERLOADED_APIS dictionary, ALLOW_MISSING_DIFF_DOCS list, api_diff_map for O(1) lookup, optimized validation logic |
| get_api_difference_info.py | Refactored discover_all_metas to recursively scan all .md files with automatic library prefix detection |
| transformers.PretrainedConfig.md | Deleted from torch_more_args (moved to invok_only_diff) |
| transformers.GenerationConfig.md | Deleted from torch_more_args (moved to invok_only_diff) |
| transformers.AddedToken.md | Deleted from torch_more_args (moved to invok_only_diff) |
| torchvision.models.inception.*.md | Added new InceptionA-E documentation |
| torchvision.models.Inception3.md | Added new Inception3 documentation |
| torch.*.md (multiple) | Updated API signatures with asterisks for keyword-only parameters |
| transformers.*.md (multiple) | Changed references from paddlenlp to paddleformers |
| torch.nn.Module.*.md (multiple) | Deleted files moved from paddle_more_args/input_args_usage_diff to other categories |
zhwesky2010
reviewed
Jan 13, 2026
Collaborator
zhwesky2010
left a comment
zhwesky2010
left a comment
There was a problem hiding this comment.
- 有些不符合 差异文档规范 的，需要按照这个文档规范来 pytorch_api_mapping_format_cn.md，CI上有自动化工具拦截不符合规范
- 可能有一个问题：paconvert中有些no_need_convert还没更新，最近又修改了不少api，可能得先测下paconvert，将已对齐的都加上去
...ides/model_convert/convert_from_pytorch/api_difference/composite_implement/torch.std_mean.md
Outdated
Show resolved
Hide resolved
.../guides/model_convert/convert_from_pytorch/api_difference/args_name_diff/torch.Tensor.std.md
Outdated
Show resolved
Hide resolved
...ides/model_convert/convert_from_pytorch/api_difference/composite_implement/torch.var_mean.md
Outdated
Show resolved
Hide resolved
...s/model_convert/convert_from_pytorch/api_difference/input_args_type_diff/torch.block_diag.md
Outdated
Show resolved
Hide resolved
..._convert/convert_from_pytorch/api_difference/input_args_type_diff/torch.broadcast_tensors.md
Outdated
Show resolved
Hide resolved
...ides/model_convert/convert_from_pytorch/api_difference/torch_more_args/torch.Tensor.round.md
Outdated
Show resolved
Hide resolved
...des/model_convert/convert_from_pytorch/api_difference/torch_more_args/torch.Tensor.round_.md
Outdated
Show resolved
Hide resolved
docs/guides/model_convert/convert_from_pytorch/api_difference/torch_more_args/torch.baddbmm.md
Outdated
Show resolved
Hide resolved
...el_convert/convert_from_pytorch/api_difference/torch_more_args/torch.nn.functional.kl_div.md
Outdated
Show resolved
Hide resolved
| # functions currently. Currently, we hard code the check of overloaded functions | ||
| # in this file. | ||
|
|
||
| OVERLOADED_APIS = { |
Collaborator
There was a problem hiding this comment.
这个名单怎么这么长，这个是不是可以加到pre-commit白名单里
Contributor
Author
There was a problem hiding this comment.
这个名单怎么这么长
移除了可以合并为一个签名的重载。
这个是不是可以加到pre-commit白名单里
感觉没必要吧，格式化后可读性可好一点
Collaborator
There was a problem hiding this comment.
这个名单怎么这么长
移除了可以合并为一个签名的重载。
这个是不是可以加到pre-commit白名单里
感觉没必要吧，格式化后可读性可好一点
白名单只需要能跑就行，一般不需要可读性，不用占太多行或者单独挪一个文件。
zhwesky2010
reviewed
Jan 15, 2026
.../guides/model_convert/convert_from_pytorch/api_difference/args_name_diff/torch.Tensor.var.md
Show resolved
Hide resolved
| paddleformers.generation.LogitsProcessor(input_ids: paddle.Tensor, scores: paddle.Tensor) | ||
| ``` | ||
|
|
||
| 两者功能一致但参数名不一致，部分参数名不同，具体如 |
| torch.std_mean(input, dim, unbiased=True, keepdim=False) | ||
| torch.std_mean(input, dim=None, unbiased=True, keepdim=False, *, correction=None) | ||
| ``` | ||
| 用于实现返回 Tensor 的标准差和均值，PaddlePaddle 目前暂无对应 API，可使用如下代码组合实现该 API。 |
Collaborator
There was a problem hiding this comment.
这种格式来写：
注：torch旧版本额外重载了
torch.std_mean(input, unbiased=True)的签名，该用法未提供转写示例
| ```python | ||
| # PyTorch 写法 | ||
| std, mean = torch.std_mean(x, dim=1) | ||
| std, mean = torch.std_mean(x, True) # torch 支持 unbiased 以第二个位置参数的形式传入 |
| ```python | ||
| # PyTorch 写法 | ||
| var, mean = torch.var_mean(x, dim=1) | ||
| var, mean = torch.var_mean(x, True) # torch 支持 unbiased 以第二个位置参数的形式传入 |
| ## [ 仅 API 调用方式不一致 ]transformers.LogitsProcessorList | ||
| ### [transformers.LogitsProcessorList](https://hf-mirror.com/docs/transformers/v4.42.0/en/internal/generation_utils#transformers.LogitsProcessorList) | ||
| ```python | ||
| transformers.LogitsProcessorList() |
Collaborator
There was a problem hiding this comment.
形参有吗，太多的话写成 **kwargs，不然容易被用户当做无参函数。
| ```python | ||
| torch.linalg.matrix_rank(A, *, atol=None, rtol=None ,hermitian=False, out=None) | ||
| torch.linalg.matrix_rank(x, tol=None, hermitian=False, *, name=None) | ||
| torch.linalg.matrix_rank(x, tol=None, hermitian=False, *, out=None) |
Collaborator
There was a problem hiding this comment.
TypeError: linalg_matrix_rank() received an invalid combination of arguments - got (), but expected one of:
* (Tensor input, *, Tensor atol = None, Tensor rtol = None, bool hermitian = False, Tensor out = None)
* (Tensor input, *, float atol = None, float rtol = None, bool hermitian = False, Tensor out = None)
* (Tensor input, Tensor tol, bool hermitian = False, *, Tensor out = None)
* (Tensor input, float tol, bool hermitian = False, *, Tensor out = None)
| ```python | ||
| torch.linalg.matrix_rank(A, *, atol=None, rtol=None ,hermitian=False, out=None) | ||
| torch.linalg.matrix_rank(x, tol=None, hermitian=False, *, name=None) | ||
| torch.linalg.matrix_rank(x, tol=None, hermitian=False, *, out=None) |
| | inception_blocks | - | 用于构建网络的 Inception 模块，Paddle 无此参数，暂无转写方式。 | | ||
| | init_weights | - | 是否对权重进行初始化，Paddle 无此参数，暂无转写方式。 | | ||
| | dropout | - | Dropout 概率，Paddle 无此参数，暂无转写方式。 | | ||
| | - | with_pool | 是否在最后的全连接层前使用池化，Paddle 特有参数。 | |
| # functions currently. Currently, we hard code the check of overloaded functions | ||
| # in this file. | ||
|
|
||
| OVERLOADED_APIS = { |
Collaborator
There was a problem hiding this comment.
这个名单怎么这么长
移除了可以合并为一个签名的重载。
这个是不是可以加到pre-commit白名单里
感觉没必要吧，格式化后可读性可好一点
白名单只需要能跑就行，一般不需要可读性，不用占太多行或者单独挪一个文件。
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
本 PR 修复差异文档与 PaConvert 中映射规则不一致的问题，主要涉及以下几类