Fix KernelAgent backend integration issues

[jiannanWang]

This PR mainly solves KernelAgent integration issues with BackendBench.

Key changes:

• Centralized Kernel Generation: Adds a method to generate kernels for all specified operators, with per-op summaries and an overall report.
• Consistent Operator Name: Extracts the correct folder names using a util function.
• Organized Kernel Files: The generated kernel files are now grouped by operator and aligned with the directory backend.
• Operator Filtering (torchbench suite): Switches to exact name matching to avoid false positives. (e.g., --ops relu.default now test only aten.relu.default, not aten.leaky_relu.default)

Example Usage:
Command:
uv run python BackendBench/scripts/main.py --suite torchbench --backend kernel_agent --ops relu.default,leaky_relu.default --topn 5

Result

# KernelAgent logs are skipped

correctness score (mean pass rate over all operators): 1.00
performance score (geomean speedup over all operators): 1.00
perf@p score (rate of correct samples with a speedup greater than p, p=1.0): 0.50

Folder Structure Example: (some files are skipped for simplicity.)

kernel_agent_run_20260114_152026
├── agent_logs
│   └── ...
├── leaky_relu__default
│   ├── leaky_relu__default_implementation_v1.py
│   └── leaky_relu__default_summary.txt
├── leaky_relu__default_session_session_20260114_152046_533793
│   └── ...
├── relu__default
│   ├── relu__default_implementation_v1.py
│   └── relu__default_summary.txt
├── relu__default_session_session_20260114_152250_994449
│   └── ...
├── OVERALL_SUMMARY.txt
└── README.md

[msaroufim]

so we had some existing setup directory utils IIRC, wouldn't this just duplicate stuff?

[jiannanWang]

We do have a similar function in the llm backend and this function is a duplicate for that.

This kernel_file_path is only used in the llm and kernel_agent backends because those backends requires generating kernels.

Since kernel_file_path is only used in the llm and kernel_agent backends, I could create a new class, GenerativeBackend, which inherits from Backend. I can move the shared functions (e.g., _generate_kernel_file_path) from both llm and kernel_agent into GenerativeBackend. Then, both the LLM and kernel_agent classes can inherit from GenerativeBackend instead of directly from Backend, which will help reduce duplication.

[msaroufim]

would it not be possible to have the same function?

[msaroufim]

this function means something totally different now, we went from something like write_kernel_to_file to bind the compiled op? so maybe the function just needs a better name

[msaroufim]

what is this doing? I'd have expected a daemon to be a service and not a boolean. Also generally boolean args in a function typically hint that you might need 2 seperate functions

[msaroufim]

I feel like a few stuff is getting mixed here

1. What the eval is expected to output
2. What kernelagent is expected to output
3. What the specific integration of kernelagent and the eval is supposed to follow

For instance Sahan also had some extensive reporting code in the past, so I'd like to hear some ideas for how to clean all this up