Skip to content

Add @perf_event program type with full attach/detach/count support#18

Open
SiyuanSun0736 wants to merge 2 commits intomultikernel:mainfrom
SiyuanSun0736:perf_pr
Open

Add @perf_event program type with full attach/detach/count support#18
SiyuanSun0736 wants to merge 2 commits intomultikernel:mainfrom
SiyuanSun0736:perf_pr

Conversation

@SiyuanSun0736
Copy link
Copy Markdown
Contributor

Summary

This PR adds first-class support for @perf_event eBPF programs in KernelScript. Users can now attach eBPF logic to hardware and software performance counters (branch misses, CPU cycles, cache misses, etc.) with a clean, safe, single-file workflow — no manual perf_event_open(2) boilerplate required.


New language features

@perf_event program attribute

@perf_event
fn on_branch_miss(ctx: *bpf_perf_event_data) -> i32 {
    return 0
}

perf_event_attr struct literal — passed directly to attach():

var attr = perf_event_attr {
    counter: branch_misses,
    pid: -1,        // all processes
    cpu: 0,         // CPU 0
    period: 1000000,
    wakeup: 1,
    inherit: false,
    exclude_kernel: false,
    exclude_user: false
}
var prog = load(on_branch_miss)
attach(prog, attr)    // two-argument perf_event form
detach(prog)

perf_counter enum — 9 hardware/software counters:
cpu_cycles, instructions, cache_references, cache_misses, branch_instructions, branch_misses, page_faults, context_switches, cpu_migrations


Compiler & codegen changes

Component Change
ast.ml PerfEvent program type, perf_event_attr struct, perf_counter enum
type_checker.ml Validates @perf_event signature and perf_event_attr field types
ir_generator.ml Recognises PerfEvent program type
ebpf_c_codegen.ml Emits SEC("perf_event") and bpf_perf_event_data context
context/perf_event_codegen.ml New module for perf event context field access codegen
userspace_codegen.ml Full perf_event lifecycle codegen (see below)
stdlib.ml attach/detach builtin resolution for perf_event form
btf_parser.ml perf_event_attr BTF type extraction

Generated userspace C functions (emitted when attach(prog, attr) is used):

  • ks_open_perf_event(ks_perf_event_attr) — maps perf_counter enum → PERF_TYPE_*/PERF_COUNT_*, validates pid/cpu rules, calls syscall(SYS_perf_event_open, ...)
  • ks_read_perf_count(int perf_fd)int64_t — reads raw counter value via read()
  • ks_print_perf_count(int perf_fd, const char *name) — prints [perf] <name>: <count> with PRId64

Attach sequence (compiler-generated, in order):

  1. attr.disabled = 1 — open without starting
  2. perf_event_open(2)perf_fd
  3. IOC_RESET — zero the counter before BPF attachment
  4. bpf_program__attach_perf_event(prog, perf_fd) — link BPF program
  5. IOC_ENABLEstart counting

Detach sequence: IOC_DISABLEbpf_link__destroyclose(perf_fd)

pid/cpu validation (enforced at runtime):

pid cpu Result
≥ 0 ≥ 0 OK — specific process, specific CPU
≥ 0 -1 OK — specific process, any CPU
-1 ≥ 0 OK — all processes, specific CPU (system-wide)
-1 -1 Error — rejected

Tests

New file test_perf_event_attach.ml — 6 test cases:

Test What it verifies
perf_event_codegen_enforces_pid_cpu_rules pid/cpu validation, IOC_ENABLE/DISABLE, success messages
perf_event_counting_starts_correctly disabled=1IOC_RESETattach_perf_eventIOC_ENABLE ordering
perf_event_period_and_wakeup_defaults safe defaults (1000000/1) substituted when period/wakeup = 0
perf_event_period_and_wakeup_custom runtime conditional expression emitted for custom values
perf_read_count_function_generated ks_read_perf_count and ks_print_perf_count generated correctly
standard_attach_uses_libbpf_error_checks all attach branches use libbpf_get_error

All 6 tests pass. perf_branch_miss.ks compiles successfully end-to-end.


Documentation

  • README.md@perf_event added to program types overview; new Hardware Performance Counter Programs section with full lifecycle example and perf_counter table
  • SPEC.md — new § 3.1.3 Perf Event Programs: syntax, pid/cpu rules, perf_counter enum, generated C helpers table, attach/detach sequences, compiler implementation notes

Testing checklist

  • dune build — passes
  • dune build @tests — all OCaml unit tests pass
  • test_all_examples.sh — perf_branch_miss.ks compiles KS→C→binary successfully (remaining failures are pre-existing and unrelated to this PR)

Copilot AI review requested due to automatic review settings April 25, 2026 14:48
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds first-class @perf_event support to KernelScript, enabling eBPF programs to attach to hardware/software performance counters via a perf_event_attr literal and a dedicated attach(handle, attr) workflow with compiler-generated userspace lifecycle helpers.

Changes:

  • Introduces PerfEvent program type end-to-end (AST/type-checking/IR/codegen) with SEC("perf_event") and *bpf_perf_event_data context.
  • Extends attach() to support a 2-arg perf-event form, generating userspace perf_event_open + attach/enable/disable + read/print helpers.
  • Adds tests, example, and documentation for perf event programs, counters, and lifecycle semantics.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
tests/test_program_ref.ml Updates stdlib integration expectations for attach() custom validation and error assertion.
tests/test_perf_event_attach.ml Adds unit tests covering perf attach/detach sequencing, pid/cpu validation, defaults, and helper generation.
tests/test_ir.ml Extends IR program type pretty-printing for PerfEvent.
tests/dune Registers new perf-event attach test executable and runtest rule.
src/userspace_codegen.ml Adds perf-event attach detection, conditional headers/types, perf fd lifecycle helpers, and perf attach/detach support.
src/type_checker.ml Adds @perf_event program type recognition and partial signature validation.
src/stdlib.ml Adds perf_counter / perf_event_attr builtins and custom attach() validation for 2-arg/3-arg forms.
src/multi_program_analyzer.ml Adds execution context and discovery logic for PerfEvent programs.
src/main.ml Adds perf_event to init program-type whitelist and project description text.
src/ir_generator.ml Adjusts lowering for non-void function calls used as statements (emit IRCall with discarded return).
src/ir_function_system.ml Adds perf-event entrypoint signature validation at IR validation stage.
src/ebpf_c_codegen.ml Registers perf-event context codegen and emits SEC("perf_event") / perf-event context detection.
src/context/perf_event_codegen.ml New perf-event context codegen module (includes/field access/section name).
src/context/dune Adds perf_event_codegen module to the context library build.
src/codegen_common.ml Maps perf_event_attr IR struct to ks_perf_event_attr in generated C.
src/btf_parser.ml Adds perf-event program template generation and BTF type list updates.
src/ast.ml Adds PerfEvent to program_type and string conversion.
examples/perf_branch_miss.ks New example demonstrating @perf_event attach/detach with perf_event_attr.
SPEC.md Documents perf-event syntax, pid/cpu rules, counters, helper generation, and attach/detach sequences.
README.md Adds perf-event overview, lifecycle example, and counter table.
BUILTINS.md Documents attach(handle, attr) overload and usage example.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/ir_function_system.ml Outdated
Comment thread src/ebpf_c_codegen.ml
Comment thread src/type_checker.ml
Comment thread src/userspace_codegen.ml
Comment thread src/userspace_codegen.ml Outdated
## Language changes
- New `@perf_event` program attribute; context type `*bpf_perf_event_data`
- New `perf_event_attr` struct literal with counter, pid, cpu, period, wakeup,
  inherit, exclude_kernel, exclude_user fields
- New `perf_counter` enum: cpu_cycles, instructions, cache_references,
  cache_misses, branch_instructions, branch_misses, page_faults,
  context_switches, cpu_migrations
- `attach(prog, attr)` two-argument form for perf_event programs
- `detach(prog)` cleans up BPF link, disables and closes perf fd

## Compiler / codegen
- AST: PerfEvent program type, perf_event_attr struct, perf_counter enum
- Type checker: validates @perf_event function signatures and attr fields
- IR generator: recognises PerfEvent program type
- ebpf_c_codegen: emits SEC("perf_event") and bpf_perf_event_data context
- userspace_codegen:
  - ks_open_perf_event(): maps perf_counter enum to PERF_TYPE/PERF_COUNT
    constants, validates pid/cpu rules, calls perf_event_open(2)
  - attach sequence: disabled=1 → IOC_RESET → attach_perf_event → IOC_ENABLE
  - detach sequence: IOC_DISABLE → bpf_link__destroy → close(perf_fd)
  - ks_read_perf_count(): reads raw 64-bit counter via read()
  - ks_print_perf_count(): prints "[perf] <name>: <count>" with PRId64

## Tests
- tests/test_perf_event_attach.ml (6 test cases):
  - pid/cpu validation rules enforced
  - counting startup ordering (RESET before ENABLE, attach before ENABLE)
  - period/wakeup default values when 0
  - custom period/wakeup runtime expressions
  - ks_read/print_perf_count helpers generated with correct logic
  - standard attach branches use libbpf_get_error

## Example
- examples/perf_branch_miss.ks: minimal @perf_event example (branch misses)
- examples/perf_branch_miss/: pre-built reference C output

## Docs
- README.md: @perf_event in program types overview; perf_counter table;
  Hardware Performance Counter Programs section with full lifecycle example
- SPEC.md: section 3.1.3 Perf Event Programs — syntax, pid/cpu rules,
  perf_counter enum, generated C helpers, attach/detach sequence steps
@congwang-mk
Copy link
Copy Markdown
Contributor

congwang-mk commented May 5, 2026

Reviewed the implementation; the codegen and tests are solid. Wanted to raise a higher-level design concern about the user-facing API before this lands, since changing it after merge would be a breaking change.

Concerns with the current API

  1. perf_event_attr collides with the kernel struct of the same name in <linux/perf_event.h>. The PR works around this in codegen_common.ml by renaming the IR type to ks_perf_event_attr. When you have to rename your own type to avoid your own headers, that's a sign the name is wrong — and it oversells what the type is, since Linux's perf_event_attr has 40+ fields and KS's has 8.

  2. All 8 fields are required for every call site. Even the trivial "count branch misses on CPU 0" case forces the user to write inherit: false, exclude_kernel: false, exclude_user: false. The codegen already substitutes defaults for period=0 and wakeup=0, but the language surface doesn't expose that.

  3. attach(prog, attr) 2-arg form breaks the universal attach(prog, target, flags) shape that XDP/TC/kprobe/tracepoint all share. perf_event becomes the only program type with a different attach signature. This also forced the __PERF_RAW_EMIT__ sentinel hack in userspace_codegen.ml to thread multi-statement C code through what's normally a function call.

  4. The perf_fd is hidden from KS code. The PR generates ks_read_perf_count and ks_print_perf_count C helpers, but they take an int perf_fd that the user never gets — attach() swallows it into the attachment table. The helpers exist but are uncallable from KernelScript.

Proposed design

@perf_event
fn on_branch_miss(ctx: *bpf_perf_event_data) -> i32 {
    return 0
}

fn main() -> i32 {
    var prog = load(on_branch_miss)

    // Minimal — defaults handle pid=-1, cpu=0, period=1_000_000, wakeup=1, all flags=false
    attach(prog, perf_options { counter: branch_misses }, 0)

    // ... workload ...
    print(perf_read(prog))  // observe the counter via the program handle

    detach(prog)
    return 0
}

CPU-pinned + period override:

attach(prog, perf_options {
    counter: cache_misses,
    cpu: 2,
    period: 500_000,
    exclude_kernel: true,
}, 0)

Four moves

  1. Rename perf_event_attrperf_options (or perf_config). Kills the ks_perf_event_attr codegen hack and signals "options bag" rather than "kernel struct."

  2. Make all fields except counter optional, with defaults. Defaults: pid=-1, cpu=0, period=1_000_000, wakeup=1, inherit/exclude_*=false. The (pid=-1, cpu=-1) invalid case becomes structurally hard to reach.

  3. Keep 3-arg attach. Generalize the second arg to str(128) | perf_options. validate_attach_function already does custom validation, so this is an extension of the existing pattern rather than a new shape. Drops the __PERF_RAW_EMIT__ sentinel and unifies attach codegen.

  4. Add perf_read(prog: ProgramHandle) -> u64. Looks up entry->perf_fd in the attachment table, calls read(2). The ks_read_perf_count C helper already exists in this PR — this just exposes it. Optionally add perf_print(prog, name) for the print helper.

Semantic story

perf_options { ... } is pure data — a config recipe, like "eth0" is for XDP. No side effects.

attach(prog, opts, 0) is the side-effect site (already true today). For a perf_options target it does perf_event_open(2)IOC_RESETattach_perf_eventIOC_ENABLE, exactly like the current PR — but the user-facing surface is uniform with other program types.

detach(prog) does IOC_DISABLEbpf_link__destroyclose(perf_fd), symmetric.

perf_read(prog) reads the counter currently bound to this program handle.

What this preserves

  • 3-arg attach everywhere — one mental model for the language.
  • Symmetric attach/detach lifecycle, no perf-specific extra calls.
  • Counter readability from KS code — solves the dead-helper problem.
  • No struct-literal-with-hidden-side-effects (attach is the side-effect site, which it already is).
  • The kernel's actual 1-perf_fd-to-1-BPF-program model — attach/detach opening fresh fds matches exactly what bcc/bpftrace/libbpf-tools already do per-CPU.

Prerequisite

Partial struct literals (omitted fields use declared defaults). This is a generally useful language feature, not perf-specific — anything with options bags benefits.


Happy to discuss tradeoffs. The codegen and tests in this PR are sound; this is purely a user-facing surface concern.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants