Skip to content

[Cherry-Pick][Metrics] Support cpu-cache-block-num (#6390)#6391

Draft
Copilot wants to merge 2 commits intorelease/2.4from
copilot/cherry-pick-pr-6390-to-release-2-4
Draft

[Cherry-Pick][Metrics] Support cpu-cache-block-num (#6390)#6391
Copilot wants to merge 2 commits intorelease/2.4from
copilot/cherry-pick-pr-6390-to-release-2-4

Conversation

Copy link
Contributor

Copilot AI commented Feb 7, 2026

Motivation

Cherry-pick PR #6390 to release/2.4 branch. Adds max_cpu_block_num metric to report CPU cache block capacity alongside existing GPU block metrics.

Modifications

Core changes:

  • fastdeploy/metrics/metrics.py: Added max_cpu_block_num Gauge metric definition to MetricsManager and METRIC_CONFIGS
  • fastdeploy/cache_manager/prefix_cache_manager.py: Set max_cpu_block_num metric in __init__() and update_cache_config() methods using self.num_cpu_blocks

Documentation:

  • docs/online_serving/metrics.md: Added fastdeploy:max_cpu_block_num metric entry
  • docs/zh/online_serving/metrics.md: Added metric entry and clarified max_gpu_block_num description to specify GPU blocks

Usage or Command

The new metric is automatically reported when FastDeploy starts with CPU cache enabled:

fastdeploy:max_cpu_block_num (Gauge) - Total number of CPU blocks initialized at startup

Accuracy Tests

Documentation-only change for metrics reporting. No impact on model outputs or computation.

Checklist

  • Add at least a tag in the PR title.
  • Format your code, run pre-commit before commit.
  • Add unit tests. No new tests needed - cherry-pick of already-reviewed code from develop branch.
  • Provide accuracy results. N/A - metrics reporting only, no model changes.
  • If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.
Original prompt

@PaddlePaddle/FastDeploy/pull/6390 这个PR Cherry pick至release/2.4分支


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@paddle-bot
Copy link

paddle-bot bot commented Feb 7, 2026

Thanks for your contribution!

Copilot AI changed the title [WIP] Cherry pick PR 6390 to release 2.4 branch [Cherry-Pick][Metrics] Support cpu-cache-block-num (#6390) Feb 7, 2026
Copilot AI requested a review from Jiang-Jia-Jun February 7, 2026 16:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants