BenchKit

BenchKit is a shell-first benchmarking framework for the CX Framework. It supports build and run workflows for multiple applications and systems, result collection, profiler data handling, estimation workflows, and result portal integration.

Origin

Repository migration details are documented in docs/repository-history.md.

Purpose

• Run benchmarks across multiple codes and systems with a shared workflow.
• Support both cross-build and native execution environments.
• Keep site-specific configuration separate from benchmark logic.
• Collect result data, profiler outputs, and estimation inputs in a consistent format.
• Integrate with the CX result portal and related estimation workflows.
• Provide a practical base for performance analysis, estimation, and feedback.

Quick Links

• Contributing guide: CONTRIBUTING.md
• Add a new application: docs/guides/add-app.md
• Add a new system: docs/guides/add-site.md
• Add estimation support: docs/guides/add-estimation.md
• CI execution control: docs/ci.md
• Security policy: SECURITY.md
• Profiler support guide: docs/guides/profiler-support.md
• Profiler level reference: docs/guides/profiler-level-reference.md

CX Framework Documents

Core specifications:

• CX framework: top-level concept and terminology.
• CX platform: system-level responsibilities and component boundaries.
• BenchKit specification: BenchKit responsibilities, interfaces, and future extension points.
• BenchKit gap analysis: implementation-facing functional gaps against the BenchKit specification.

Estimation specifications:

• Estimation: common rules for accepting, running, storing, and presenting estimation functions. Detailed estimation data, package, input, and re-estimation specifications are linked from this document.
• Result storage design: storage-design memo for result and estimate artifacts.

Operational specifications:

• Audit log specification: result-server audit log events and handling rules.

Developer Reference

The detailed developer-oriented reference has moved to docs:

• docs/guides/developer-reference.md

This includes:

• project structure
• result portal architecture
• CI pipeline structure
• configuration files
• CI execution control
• system-specific execution environments
• runtime requirements

Runtime Requirements

• result_server requires Python 3.12 or later.

License

This project is licensed under the BSD 3-Clause License. See LICENSE.