Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion plugins/agent-plugin-skills/.codex-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "agent-plugin-skills",
"version": "6.7.14",
"version": "6.7.15",
"description": "Installable maintainer skills for skills-export repositories.",
"author": {
"name": "Gale",
Expand Down
2 changes: 1 addition & 1 deletion plugins/agent-plugin-skills/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "agent-plugin-skills-maintenance"
version = "6.7.14"
version = "6.7.15"
description = "Maintainer-only Python tooling baseline for agent-plugin-skills."
requires-python = ">=3.11"
dependencies = []
Expand Down
2 changes: 1 addition & 1 deletion plugins/agent-plugin-skills/uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion plugins/apple-dev-skills/.codex-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "apple-dev-skills",
"version": "6.7.14",
"version": "6.7.15",
"description": "Apple development workflows for Codex, including SwiftUI architecture and DocC authoring guidance.",
"author": {
"name": "Gale",
Expand Down
1 change: 1 addition & 0 deletions plugins/apple-dev-skills/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,7 @@ Planned
- [ ] Decide which implementation surface should own the offload path.
- [ ] Define the input and output contract for the offload path so verification work can be delegated without losing decision-useful detail.
- [ ] Cover the highest-value offload cases first, including `swift build`, `swift test`, `xcodebuild`, preview refresh, diagnostics refresh, and noisy failure summarization.
- [ ] Cover performance-sensitive Apple silicon package and Xcode workflows, including `OSSignposter`, `xctrace`, Time Profiler, Metal System Trace, Allocations, VM Tracker, Audio, MLX, local AI, and trace artifact reporting.
- [ ] Document when the main agent should stay local versus when it should hand verification work to the offload path.
- [ ] Add validation or smoke-test coverage once the implementation surface is chosen.

Expand Down
2 changes: 1 addition & 1 deletion plugins/apple-dev-skills/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "apple-dev-skills-maintainer"
version = "6.7.14"
version = "6.7.15"
description = "Maintainer tooling for the apple-dev-skills repository"
requires-python = ">=3.9"
dependencies = []
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,15 @@ description: Guide Swift Testing, XCTest holdouts, xctestplan handoff conditions

## Purpose

Use this skill as the primary execution workflow for test-focused work in existing Swift Package Manager repositories. Keep it focused on Swift Testing, XCTest holdouts, `.xctestplan` handoff conditions, async-test guidance, semantic accessibility-test boundaries, filters, retries, fixtures, and package-level test diagnosis instead of broad manifest and build/run work. `scripts/run_workflow.py` is the runtime entrypoint for repo-shape checks, test-surface command planning, and clean handoff to the build/run or Xcode-oriented surfaces when the request drifts.
Use this skill as the primary execution workflow for test-focused work in existing Swift Package Manager repositories. Keep it focused on Swift Testing, XCTest holdouts, `.xctestplan` handoff conditions, async-test guidance, semantic accessibility-test boundaries, performance-sensitive package workload profiling, filters, retries, fixtures, and package-level test diagnosis instead of broad manifest and build/run work. `scripts/run_workflow.py` is the runtime entrypoint for repo-shape checks, test-surface command planning, and clean handoff to the build/run or Xcode-oriented surfaces when the request drifts.

## When To Use

- Use this skill for running, diagnosing, organizing, or evolving tests in an existing Swift package repo.
- Use this skill for Swift Testing-first package work, XCTest holdouts, async-test design, semantic accessibility-test boundaries, and test-fixture organization.
- Use this skill for package-level `.xctestplan` execution when the package surface needs Xcode test-plan follow-through.
- Use this skill when the request is about test selection, filtering, retries, failures, flaky tests, or test-only Debug/Release validation.
- Use this skill when the request is about package-first performance-sensitive testing, signpost placement, Release workload harnesses, or profiling-ready package test design for Audio, Metal, MLX, local AI, streaming, or other Apple silicon-sensitive workloads.
- Do not use this skill for broad manifest edits, dependency work, package resources, plugin flows, or ordinary build and run work.
- Do not use this skill for brand-new package bootstrap from nothing.
- Do not use this skill for repo-guidance alignment in an existing package repo.
Expand Down Expand Up @@ -47,8 +48,9 @@ Use this skill as the primary execution workflow for test-focused work in existi
- preserve its package-appropriate logging, telemetry, structured-concurrency, and Swift Testing guidance
4. Run `scripts/run_workflow.py` to resolve repo shape, confirm the request stays on the testing surface, and plan the package-testing command path.
5. Use `references/package-resources-testing-and-builds.md` when the request touches Swift Testing, XCTest, `.xctestplan`, accessibility-related semantic tests, fixtures, async test discipline, or test-related Debug/Release validation.
6. If the repo root is ambiguous because Xcode-managed markers are present at the same root, use `references/xcode-handoff-conditions.md` and hand off cleanly to `xcode-testing-workflow`.
7. Report which parts were agent-executed, the docs relied on, the repo-shape result, and any required next step or handoff.
6. Use `references/performance-sensitive-testing-and-profiling.md` when the request touches package-first instrumentation, `OSSignposter`, `xctrace`, Time Profiler, Metal System Trace, Allocations, VM Tracker, Audio, MLX, local AI, streaming, or other performance-sensitive Apple silicon workloads.
7. If the repo root is ambiguous because Xcode-managed markers are present at the same root, use `references/xcode-handoff-conditions.md` and hand off cleanly to `xcode-testing-workflow`.
8. Report which parts were agent-executed, the docs relied on, the repo-shape result, and any required next step or handoff.

## Inputs

Expand Down Expand Up @@ -100,6 +102,7 @@ Use this skill as the primary execution workflow for test-focused work in existi
- Xcode MCP mutation tools
- `.xctestplan` execution or package test behavior that is more authoritative through Xcode-managed Apple SDK integration
- direct test execution through Xcode-native destinations, UI testing, or `.xctestplan` handling inside an Xcode-managed workspace
- Instruments UI inspection, `.trace` artifact interpretation, or `xctrace` capture that depends on Xcode-managed schemes, destinations, app hosts, or test plans
- Recommend `apple-ui-accessibility-workflow` when the user is really asking how the UI should expose semantics to assistive technologies instead of how a package-side test should be organized.
- Hand off to `xcode-build-run-workflow` when package test work instead crosses into direct changes inside `.xcodeproj`, `.xcworkspace`, or `.pbxproj` managed scope.
- Recommend `sync-swift-package-guidance` when the request is really about repo guidance instead of execution.
Expand All @@ -120,6 +123,7 @@ Use this skill as the primary execution workflow for test-focused work in existi
- `references/workflow-policy.md`
- `references/repo-shape-detection.md`
- `references/package-resources-testing-and-builds.md`
- `references/performance-sensitive-testing-and-profiling.md`
- `references/xcode-handoff-conditions.md`

### Contract References
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,14 @@
- `xcrun --find swift` when Apple toolchain location matters
- `xcodebuild -version` when Apple-managed SDK or Xcode component state matters
- `xcodebuild -showComponent metalToolchain` when Metal toolchain availability is relevant
- `xcrun --find xctrace` when Instruments trace capture or export is relevant
- `xcrun xctrace version` when the active Instruments command-line tool version matters
- `xcrun xctrace list templates` before assuming an Instruments template exists on the current Xcode version

## Performance profiling

- `swift build -c release --product <ExecutableProduct>` before profiling an optimized package executable
- `xcrun xctrace record --template 'Time Profiler' --output traces/<name>.trace --launch -- .build/release/<ExecutableProduct> <args>` for CPU profile capture
- `xcrun xctrace record --template 'Metal System Trace' --time-limit 30s --output traces/<name>.trace --launch -- .build/release/<ExecutableProduct> <args>` for Metal CPU/GPU timeline capture
- `xcrun xctrace record --template 'Allocations' --output traces/<name>.trace --launch -- .build/release/<ExecutableProduct> <args>` for allocation capture
- Prefer profiling the built executable over profiling `swift run` when the measurement target is runtime behavior rather than build or package-driver overhead.
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,9 @@
- Use `xcodebuild` when schemes, destinations, configurations, test plans, or Apple-managed SDK and toolchain behavior matter.
- Validate Release builds intentionally when optimization or packaging can change behavior.
- Treat tagged releases as a cue to verify both the everyday Debug path and the Release artifact path before publishing.

## Performance-sensitive package workloads

- Use `performance-sensitive-testing-and-profiling.md` when a package test or executable workload needs repeatable timing, allocation, VM, CPU, GPU, Audio, Metal, MLX, local AI, streaming, or Apple silicon performance evidence.
- Keep ordinary correctness tests in Swift Testing or XCTest, but do not treat passing package tests as proof that performance-sensitive workloads have been profiled.
- Prefer package-owned signposts and Release executable harnesses before handing off to Xcode-managed Instruments or `xctrace` interpretation.
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# Performance-Sensitive Testing and Profiling

Use this reference when a Swift package contains performance-sensitive Apple-platform work, especially Audio, Metal, MLX, local AI models, streaming generation, real-time rendering, or large memory-mapped resources.

## Apple Documentation Anchors

- [`OSSignposter`](https://developer.apple.com/documentation/os/ossignposter) records signposted intervals and events through the unified logging system so Instruments can show package-defined phases on a trace timeline.
- [Xcode command-line tool reference](https://developer.apple.com/documentation/xcode/xcode-command-line-tool-reference) documents `xctrace` as the command-line tool for recording, importing, exporting, and symbolicating Instruments `.trace` files.
- [Installing the command-line tools](https://developer.apple.com/documentation/xcode/installing-the-command-line-tools) states that `xcodebuild` and `xctrace` ship with full Xcode, not the standalone Command Line Tools for Xcode package.
- [Metal developer workflows](https://developer.apple.com/documentation/xcode/metal-developer-workflows) describes Metal System Trace as the Instruments tool for CPU/GPU timeline and Metal memory analysis.
- [Analyzing the performance of your Metal app](https://developer.apple.com/documentation/xcode/analyzing-the-performance-of-your-metal-app) describes the Instruments Game Performance template and its included instruments, including Points of Interest, Time Profiler, Virtual Memory Trace, Metal Resource Events, Metal Application, and GPU.
- [Gathering information about memory use](https://developer.apple.com/documentation/xcode/gathering-information-about-memory-use) describes the Allocations instrument for heap and anonymous virtual memory allocations.

## Apple Silicon Baseline

- Treat Apple silicon Macs as the default profiling target for current and forward-looking Swift package performance guidance.
- Do not add Intel-specific tuning, benchmark interpretation, architecture switches, or fallback expectations unless the target repository explicitly declares Intel Mac support.
- Record the Apple silicon device class, chip family when known, macOS version, Xcode version, Swift version, package resolved versions, model identifier, model dtype or quantization, and workload fixture before comparing trace results.
- For Metal, MLX, and Audio workloads, treat unified memory, GPU scheduling, thermal state, and model/resource residency as first-class evidence instead of CPU time alone.
- Prefer measuring on a quiet machine. Do not run heavy package tests, model loads, build jobs, or trace captures concurrently with other expensive validation.

## Decision Model

- Use ordinary `swift test` for correctness, small unit coverage, fixture validation, and regression checks that do not depend on runtime performance.
- Use package-level performance tests or executable harnesses when the package needs repeatable timing, allocation, or resource-pressure evidence.
- Use `xctrace` or Instruments when the question is where CPU time is spent, how CPU and GPU work overlap, where heap or virtual memory grows, or how package-defined phases align with system behavior.
- Use Xcode-managed test plans when sanitizers, runtime API checking, destinations, or named test configurations are part of the repeatable performance contract.
- Use an app host when the package behavior depends on app lifecycle, audio session behavior, UI event loops, display timing, sandboxing, entitlements, or simulator/device state.

## Package Workload Shape

- Build before profiling so the trace captures the workload, not package resolution or compilation.
- Profile Release builds when optimization, inlining, specialization, ARC behavior, or Metal/MLX performance is the question.
- Keep Debug traces only when diagnosing debug-only assertions, sanitizer failures, or development-time runtime warnings.
- Prefer a dedicated executable target or test harness that performs one workload at a time with deterministic fixtures.
- Separate cold model load, warmup, steady-state execution, streaming or chunk generation, audio rendering, and teardown into distinct phases.
- For MLX and large local model work, run one heavy workload at a time unless the test is explicitly about concurrent model pressure.
- For Audio work, capture sample rate, buffer size, channel count, device or route assumptions, and whether playback is live, synthetic, or file-backed.
- For benchmark-style tests, keep fixture inputs stable and report whether results are wall-clock timings, CPU samples, GPU timeline evidence, allocation counts, resident memory, or virtual memory activity.

## Signpost Contract

- Use `OSSignposter` for durable package-defined trace intervals when profiling will be repeated or compared across runs.
- Keep signpost names stable, short, and phase-oriented so traces from different branches can be compared.
- Prefer one subsystem per package or executable harness and categories that describe the workload family, such as `audio`, `mlx`, `inference`, `streaming`, or `benchmark`.
- Avoid signposting every low-level helper call. Mark phases that a maintainer can interpret in Time Profiler, Points of Interest, Metal System Trace, Allocations, or VM views.
- Recommended interval names for Audio and MLX packages:
- `model.load`
- `model.warmup`
- `audio.decode`
- `audio.render`
- `inference.prefill`
- `inference.generate`
- `stream.chunk`
- `resource.teardown`
- Recommended event names:
- `fixture.ready`
- `first.audio.buffer`
- `first.token`
- `first.chunk`
- `playback.started`
- `playback.completed`

## Instrument Selection

- Use Time Profiler when the question is CPU call stacks, Swift runtime overhead, actor or task scheduling overhead, lock contention symptoms, expensive decoding, or non-GPU model-preparation work.
- Use Metal System Trace when the question is CPU/GPU overlap, command buffer timing, GPU scheduling, Metal resource events, GPU workload gaps, or whether compute work is actually reaching the GPU as expected.
- Use Allocations when the question is heap growth, allocation churn, retained object graphs, unexpected bridging, autorelease-like spikes, or per-phase allocation deltas.
- Use VM Tracker or Virtual Memory Trace when the question is mapped model weights, dirty memory growth, resident memory pressure, memory-mapped files, or large region behavior that heap allocation summaries do not explain.
- Use Points of Interest or signpost views to align package-defined phases with the system timeline.
- Use Processor Trace only when supported by the active Xcode/Instruments version and the user specifically needs lower-overhead Apple-silicon CPU execution evidence beyond Time Profiler.

## Command Guidance

- Verify full Xcode is active before relying on `xctrace`:

```bash
xcode-select -p
xcrun --find xctrace
xcrun xctrace version
```

- Inspect available trace templates on the current machine:

```bash
xcrun xctrace list templates
```

- Build the package workload before recording:

```bash
swift build -c release --product <ExecutableProduct>
```

- Record a built executable with Time Profiler:

```bash
xcrun xctrace record --template 'Time Profiler' --output traces/<name>.trace --launch -- .build/release/<ExecutableProduct> <args>
```

- Record a bounded run when the workload would otherwise continue indefinitely:

```bash
xcrun xctrace record --template 'Time Profiler' --time-limit 30s --output traces/<name>.trace --launch -- .build/release/<ExecutableProduct> <args>
```

- Record Metal work when the package executable or host app exercises GPU-backed work:

```bash
xcrun xctrace record --template 'Metal System Trace' --time-limit 30s --output traces/<name>.trace --launch -- .build/release/<ExecutableProduct> <args>
```

- Record allocation behavior:

```bash
xcrun xctrace record --template 'Allocations' --output traces/<name>.trace --launch -- .build/release/<ExecutableProduct> <args>
```

- Redirect target output when the harness prints structured workload metadata:

```bash
xcrun xctrace record --template 'Time Profiler' --target-stdout traces/<name>.stdout.txt --output traces/<name>.trace --launch -- .build/release/<ExecutableProduct> <args>
```

## Artifact Expectations

- Preserve `.trace` artifacts when they justify a performance fix, tuning decision, or regression report.
- Keep trace artifacts out of git unless the repository explicitly tracks performance evidence fixtures or release artifacts.
- Report the workload command, build configuration, hardware, OS, Xcode, Swift version, model/resource versions, trace path, and the specific question the trace answered.
- Do not pretend text export replaces opening Instruments for timeline-heavy Metal, GPU, or memory-pressure investigation.
- If a trace cannot be captured because `xctrace` is unavailable, Xcode is not active, privacy prompts block recording, or the template is missing, report that exact blocker.

## Handoff Rules

- Stay in `swift-package-testing-workflow` when the work is package workload design, Swift Testing/XCTest organization, signpost placement, Release package builds, or executable-harness profiling.
- Hand off to `xcode-testing-workflow` when the next step depends on Instruments UI, `.xctestplan` configurations, Xcode destinations, app-hosted test execution, or interpretation of `.trace` timelines.
- Hand off to `xcode-build-run-workflow` when the work becomes Metal toolchain setup, shader compilation, app launch profiling, entitlements, project membership, scheme configuration, or Xcode-managed build settings.
Loading