-
Notifications
You must be signed in to change notification settings - Fork 60
Open
Labels
CoredocumentationImprovements or additions to documentationImprovements or additions to documentationperformanceRelated to all issues and tasks focused on improving performanceRelated to all issues and tasks focused on improving performance
Milestone
Description
Build automated benchmarking infrastructure to measure and track model performance across devices and configurations.
Currently, benchmarking models consumes a lot of time:
- Manual testing on each device
- No standardized benchmark suite
- Results scattered across different runs/people
- No comparison between model configurations (quantization, delegates)
Goals
- Automated benchmark suite
- Standardized metrics across all models
- Historical tracking to catch performance regressions
Metadata
Metadata
Assignees
Labels
CoredocumentationImprovements or additions to documentationImprovements or additions to documentationperformanceRelated to all issues and tasks focused on improving performanceRelated to all issues and tasks focused on improving performance