Add ck-build-analysis skill for compilation profiling

[tenpercent]

Add Build Time Analysis Skill

Summary

Adds ck-build-analysis skill to automate compilation profiling using Clang's -ftime-trace feature. This skill helps identify template instantiation bottlenecks and optimize build times for Composable Kernel targets.

Motivation

CK's heavy use of template metaprogramming leads to long compilation times (20-30+ seconds per file). Understanding where compilation time is spent is critical for:

• Identifying expensive template instantiations
• Optimizing template hierarchies
• Reducing build times through explicit instantiation or extern templates
• Making informed architectural decisions

Changes

Added two new files to .claude/skills/:

1. ck-build-analysis - Executable bash script that:

   • Configures CMake with -ftime-trace and custom granularity
   • Builds the specified target in Docker
   • Analyzes the generated trace JSON file
   • Generates a comprehensive markdown report

2. ck-build-analysis.md - Documentation with:

   • Usage examples and command-line options
   • Granularity trade-offs and recommendations
   • Natural language interface for Claude

Usage

# Quick analysis (default 500µs granularity)
.claude/skills/ck-build-analysis example_convnd_fwd_xdl_fp8

# Balanced analysis (recommended - 100µs granularity)
.claude/skills/ck-build-analysis example_convnd_fwd_xdl_fp8 --granularity=100

# High-resolution analysis (1µs granularity - captures all events)
.claude/skills/ck-build-analysis example_convnd_fwd_xdl_fp8 --granularity=1 --output=detailed_report.md

# Analyze any CK target
.claude/skills/ck-build-analysis test_amdgcn_mma --granularity=100

Example Output

The generated report includes:

Executive Summary

- Wall Clock Time: 22.2 seconds
- Trace Time: 81.6 seconds
- Template Instantiation Time: 21.7 seconds (26.6% of trace)
- Total Events Captured: 26,912
- Total Template Instantiations: 13,931
- Unique Template Families: 424

Key Sections

• Compilation Phase Breakdown - Time spent in InstantiateFunction, Frontend, Backend, Optimizer
• Top 30 Individual Instantiations - Most expensive single template instantiations
• Template Families by Total Time - Which template families consume the most time
• Template Families by Count - Which templates are instantiated most frequently
• Optimization Recommendations - Actionable short/medium/long-term strategies

Analysis Results

Testing on example_convnd_fwd_xdl_fp8 revealed:

Granularity Comparison

Granularity	Trace Size	Instantiations	Unique Families
500µs (default)	3.7 MB	5,104	221
100µs (balanced)	11 MB	13,931	424
1µs (high-res)	81 MB	36,506	766

Finding: Default 500µs threshold filters out 86% of template instantiations. Using 100µs captures 2.7x more data while keeping trace files manageable.

Top Template Bottlenecks (100µs granularity)

Template Family	Count	Total Time	% of Total
TensorDescriptor	2,297	4.0s	18.5%
DeviceGroupedConvFwdMultipleABD_Xdl_CShuffle	87	2.6s	11.9%
run_grouped_conv_fwd	3	1.7s	8.1%
transform_tensor_descriptor	281	1.7s	7.7%

Key Insights:

• Template instantiation accounts for 26.6% of total compilation time
• run_grouped_conv_fwd has only 3 instantiations but averages 583ms each
• Top 10 template families account for 64% of instantiation time
• 13,931 total instantiations suggest significant over-templating

Integration

• Works with ck-docker: Automatically uses existing Docker container
• Granularity options: 500µs (fast), 100µs (balanced), 1µs (complete)
• Custom output: Specify report filename with --output
• No code changes: Pure analysis tool, doesn't modify source

Testing

Tested with multiple targets and granularity levels:

• ✅ example_convnd_fwd_xdl_fp8 (500µs, 100µs, 1µs)
• ✅ Trace file parsing and analysis
• ✅ Report generation with all statistics
• ✅ Container auto-start integration

Future Improvements

Potential enhancements:

• Compare multiple builds to track improvements
• Generate flamegraphs from trace data
• Integration with CI to track compilation time regressions
• Automated suggestions for template optimization

Example Report Preview

## Template Families by Total Time (Top 10)

| Rank | Template Family | Count | Total (ms) | Avg (ms) | % of Total |
|------|-----------------|-------|------------|----------|------------|
|    1 | TensorDescriptor                            |  2297 |    4028.86 |     1.75 |      18.5% |
|    2 | tensor_operation::device::DeviceGroupedC... |    87 |    2585.59 |    29.72 |      11.9% |
|    3 | run_grouped_conv_fwd                        |     3 |    1749.93 |   583.31 |       8.1% |
|    4 | transform_tensor_descriptor                 |   281 |    1689.94 |     6.01 |       7.7% |
...

Documentation

The .md file provides:

• Clear usage examples
• Granularity trade-off table
• Natural language interface for Claude
• Environment variable configuration
• Tips for choosing the right granularity

---

This skill enables data-driven optimization of CK build times by making -ftime-trace analysis easy and automated.